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PREFACE 


This  report  has  been  issued  in  two  volumes.  IDA  Paper 
P-1537,  Part  A,  contains  detailed  background  information  on 
DROLS  operations  and  is  primarily  intended  for  those  readers 
who  are  not  so  familiar  with  DROLS.  This  volume.  Part  B,  dis¬ 
cusses  the  research  performed  by  IDA  in  response  to  the  task 
statement . 
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I.  INTRODUCTION 


The  Defense  Technical  Information  Center  (DTIC),  under  the 
overall  direction  of  the  Office  of  the  Deputy  Under  Secretary 
of  Defense  (Research  and  Advanced  Technology),  OUSDRE,  operates 
a  DoD-wide  on-line  system  that  provides  remote  interaction  with 
four  major  RDT&E  data  bases.  The  system  was  originally  designed 
in  the  late  1960s  to  provide  secure  access  to  a  computerized  file 
containing  classified  descriptions  of  ongoing  research  efforts. 
This  original  file  of  Work  Unit  Information  System  data  was 
set  up  as  a  management  information  system  (MIS)  intended  to  help 
keep  DoD  managers  informed  about  ongoing  DoD  scientific  and  tech¬ 
nical  projects.  Such  a  system  would  help  to  avoid  duplication 
of  technical  effort,  help  managers  and  research  personnel  to 
stay  knowledgeable  about  rapidly  advancing  science  and  technology 
and  enable  them  to  borrow  and  use  ideas  from  each  other  that  might 
be  relevant  to  their  projects. 

Once  the  system  was  in  place  and  operating,  it  became  ap¬ 
parent  that  it  could  have  a  much  broader  application  than  the 
one  for  which  it  was  expressly  designed.  Three  data  bases  were 
added:  The  Technical  Reports  (TR),  Research  and  Development 

Program  Planning  (PP),  and  Independent  Research  and  Development 
(IR&D)  files.  The  operational  system  became  known  as  the  Defense 
RDT&E  On-Line  System  (DROLS). 

Of  the  four  files  the  TR  file  is  by  far  the  largest,  con¬ 
taining  about  1.3  million  entries.  Also,  as  might  be  expected, 
the  TR  file  is  the  most  frequently  queried.  Although  it  was 
designed  and  constrained  to  protect  a  small  proportion  (about 
10  percent  of  intrinsically  classified  records,  DROLS  is  now 


very  different  in  scope  and  intent  from  its  original  implemen¬ 
tation  with  the  work  unit  file.  Because  of  what  seem,  from  this 
time  vantage  point,  to  be  arbitrary  design  decisions,  DROLS  is 
not  compatible  with  any  other  DoD  systems  for  electronic  access 
to  or  transfer  of  S&T  information.  DTIC  cannot,  for  instance, 
interface  with  and  utilize  the  computer  resources  sharing  and 
terminal  access  capabilities  of  the  DoD-developed  ARPANET.  It 
has  difficulty  taking  advantage  of  the  many  software  and  hard¬ 
ware  developments  in  the  ADP  marketplace.  These  deficiencies 
are  manifested  by  high  costs  for  operation,  access  and  enhance¬ 
ment  . 


DTIC  has  found  some  ways  of  circumventing  the  original 
limitations  of  DROLS.  This  year,  for  example,  it  has  developed 
software  that  permits  unclassified  users  to  access  unclassified 
fields  that  pertain  to  classified  reports  in  the  file.  In  ad¬ 
dition,  it  is  now  possible  for  low-speed  asynchronous  dial-up 
terminals  to  access  the  unclassified  DROLS  files.  Since  this 
study  was  started,  these  and  other  improvements  have  been  made 
to  DROLS  that  effectively  enhance  unclassified  users’  capability 
to  obtain  almost  as  much  information  from  the  files  as  the 
cleared  users.  We  believe  that  DTIC  has  gone  about  as  far  as  it 
can  in  enhancing  the  utility  of  the  unclassified  terminals  while 
still  conforming  to  its  mandate  to  protect  classified  information 
by  releasing  it  only  to  those  who  are  properly  cleared  to  receive 
it.  It  has  been  able  to  accomplish  what  is  has  done  so  far, 
partly  because  only  a  small  portion  of  the  information  in  its 
files  is  classified  and  even  of  those  reports  that  are  clas¬ 
sified,  a  number  of  fields  such  as  title,  AD  number,  authors  and 
descriptors  are  not  classified. 

However,  the  protection  of  a  small  amount  of  classified 
information  complicates  further  improvement  of  the  system.  It 
also  maintains  the  high  operating  costs  associated  with  secure 
operating  systems. 
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In  October  1979,  during  a  ceremony  commemorating  the  re¬ 
designation  of  the  Defense  Documentation  Center  to  Defense  Tech¬ 
nical  Information  Center  (DTIC),  Dr.  Ruth  M.  Davis,  then  the 
Deputy  Under  Secretary  of  Defense  for  Research  and  Engineering 
(R&AT),  addressed  DTIC  on  the  subject  of  its  role.  Dr.  Davis 
said  that  DTIC  should  expect  to  "work  more  closely  with  R&D 
managers  in  the  Pentagon  and  provide  them  with  technical  and 
management  information."  In  addition,  she  recommended  that  DTIC 
"play  a  more  important  role  in  the  evolution  of  OUSDRE  data  pro¬ 
cessing  activities  . . .  and  assisting  and  improving  our  management 
information  systems  as  well."  DTIC  should  "become  part  of  a 
community  providing  internal  and  external  interactive  data  pro¬ 
cessing  support  to  OSD."  It  should  have  "R&D  capability  to  take 
care  of  its  own  growing  needs  and  the  requirements  of  the  Defense 
STI  program."  DTIC  in  the  future  will  "contribute  to  the  edu¬ 
cation  and  training  of  R&D  managers  in  the  use  of  DTIC  and  other 
data  bases  and  in  generating  a  Federal-wide  information  inter¬ 
change  program.  It  will  also  encourage  the  improvement  of  the 
quality  of  technical  reports..."  (Ref.  1). 

In  view  of  these  changes  in  scope  and  intent,  it  seems  rea¬ 
sonable  to  examine  the  purposes  and  priorities  of  DTIC  insofar 
as  they  concern  the  handling  of  classified  information  on-line. 
For  example,  considering  the  latest  developments  in  information 
technology,  would  there  be  a  net  advantage  to  DTIC  and  the  Com¬ 
munity  it  serves,  as  well  as  the  Department  of  Defense,  if  DROLS 
became  a  completely  unclassified  system?  What  alternatives  are 
available  to  DTIC  to  supply  the  needs  of  the  community  that  uses 
classified  information,  if  DROLS  were  limited  to  unclassified 
information?  What  would  be  users  attitude  toward  such  a  change? 
What  alternative  courses  of  action  do  users  have  if  they  can  no 
longer  obtain  classified  citations  from  DTIC  on-line? 


3 


In  this  study  we  have  tried  to  examine  these  questions.  We 
have  had  to  remind  ourselves  continually  that  DROLS  does  not  have 
in  its  banks  much  that  would  be  considered  by  scientists  and  en¬ 
gineers  as  "real"  data.  DROLS  provides  primarily  bibliographic 
information  and  management  information  such  as  who  is  working  on 
what.  DROLS  then  is  really  more  a  source  of  where  to  find  it 
rather  than  of  information  and  data.  It  is  not  the  only  such 
source  that  DTIC  provides,  but  it  is  the  most  convenient,  time 
saving  and  comprehensive  source  of  where  to  find  it  information 
that  DTIC  offers.  Unfortunately,  although  only  a  small  part  of 
this  information  is  classified,  it  is  necessary  that  the  entire 
system  be  set  up  to  protect  this  small  proportion  of  classified 
information.  Hence,  all  information  that  goes  out  to  classified 
terminals  must  be  encrypted  and  all  such  terminals  must  be  lo¬ 
cated  in  appropriate  enclosures  and  attended  by  properly  cleared 
personnel.  In  addition,  strict  controls  must  be  maintained  on 
user  protocols  for  addressing  the  system  to  ensure  that  users 
receive  only  the  information  they  are  permitted  to  receive  by 
virtue  of  their  security  status  and  need  to  know. 

DTIC  asked  IDA  to  provide  an  examination  and  analysis  that 
might  help  to  improve  access  to  DROLS.  Of  particular  interest 
was  the  question  of  whether  DROLS  should  become  a  totally  un¬ 
classified  system,  with  classified  user's  needs  handled  by  some 
other  means.  To  examine  this  question  IDA  was  asked  to  examine 
the  basic  objectives  and  priorities  for  providing  on-line  access 
to  and  communications  of  relevant  DTIC-stored  Scientific  and 
Technical  Information  (STI)  in  support  of  Defense  RDT&E  programs 
and  also  the  National  science  and  technology  community. 

This  report  describes  the  work  and  results  of  this  IDA 
effort,  which  was  initiated  on  December  10,  1979*  During  the 
course  of  this  work,  DTIC  initiated  a  number  of  enhancements  of 
DROLS  that  improved  considerably  the  service  to  the  unclassified 
user.  Some  of  these  improvements  came  unexpectedly,  and,  since 
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some  of  the  subtasks  were  made  obsolete  by  the  improvements,  the 
study  task  was  redirected  by  mutual  agreement.  In  this  report 
we  have  attempted  to  indicate  the  work  that  was  accomplished  in 
response  to  the  original  task  statement  as  well  as  to  the  re¬ 
direction. 
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II.  SUMMARY  AND  CONCLUSIONS 

In  December  1979 ,  the  Defense  Technical  Information  Center 
(DTIC)  requested  the  Institute  for  Defense  Analyses  (IDA)  to 
undertake  a  study  of  its  remote  terminal  accessible  bibliographic 
reference  system  known  as  the  Defense  RDT&E  On-Line  System  (DROLS). 
The  requested  study  was  to  consider  the  principal  objectives  and 
priorities  involved  in  the  operation  and  design  of  on-line  scien¬ 
tific  and  technological  data  base  systems  in  support  of  the  De¬ 
fense  RDT&E  community,  to  review  the  current  secure  communications 
service  used  by  DROLS,  to  survey  a  sample  of  DROLS  users  to  obtain 
their  reactions  to  the  proposition  that  DROLS  become  an  unclassi¬ 
fied  system,  to  examine  the  need  and  utility  of  the  classified 
information  available  from  DROLS  and  to  provide  some  cost  infor¬ 
mation  relative  to  classified  and  unclassified  terminals. 

Our  examination  of  the  applicable  regulations,  directives 
and  instructions  revealed  a  long  list  of  documents  whose  intention 
is  to  make  DTIC  a  central  repository  of  Defense  RDT&E  information. 
The  regulations  also  charge  DTIC  with  the  requirement  to  protect 
all  such  information  that  may  be  classified.  There  are  some 
limits  to  the  extent  of  DTIC ' s  domain.  For  example,  it  is  not 
authorized  to  have  any  information  classified  above  the  "Secret" 
level,  nor  is  it  to  have  any  communications  security  or  intelli¬ 
gence  information  in  its  files.  In  addition,  DTIC  is  required  to 
limit  access  to  any  information  that  might  be  classed  as  limited 
distribution  by  anyone,  although,  insofar  as  DROLS  is  concerned, 
this  only  prevents  some  users  from  seeing  abstracts  of  such 
documents.  The  most  significant  limitation  to  DTIC ' s  services 
and  to  DROLS  is  derived  from  another  problem — documents  that 
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should  be  submitted  are  simply  withheld;  thus,  there  Is  no  in¬ 
dication  anywhere  in  the  files  that  such  documents  exist.  We 
found  estimates  that  as  much  as  fifty  percent  of  all  documents 
that  should  be  submitted  to  DTIC  are  withheld,  thus  severely 
hampering  the  effectiveness  of  DTIC  and  of  DROLS  in  performing 
their  assigned  functions. 

In  the  past  year  DTIC,  in  making  improvements  to  DROLS,  has 
made  it  possible  to  access  the  system  and  receive  unclassified 
information  from  asynchronous  dial-up  terminals  anywhere  that 
they  can  connect  with  a  telephone  line.  Furthermore,  the  system 
has  been  changed  so  that  citations  will  be  obtained  to  almost 
all  reports  in  the  file  even  on  an  unclassified  terminal.  Thus, 
the  unclassified  terminal  will  receive  nearly  as  much  information 
as  will  the  classified  terminal.  This  latter  fact  means  little 
to  most  of  the  DROLS  users  who  have  classified  terminals,  who 
show  no  interest  in  the  possible  savings  and  better  service  to 
unclassified  terminal  users  that  might  result  from  converting 
DROLS  to  an  unclassified  system.  The  prevailing  attitude  of 
classified  terminal  users  is  that  the  information  and  time  saved 
in  obtaining  it  on-line  are  well  worth  whatever  it  costs. 

Further,  we  examined  four  possible  alternatives  that  DTIC 
might  follow  in  dealing  with  DROLS.  It  is  our  view  that  the  best 
alternative  for  now  would  be  to  keep  DROLS  substantially  as  it  is 
a  simultaneous  classified  and  unclassified  system.  The  basic 
reason  for  this  conclusion  is  that  the  classified  terminal  users 
would  object  strenuously  to  such  a  change  even  though  an  unclassi 
fied  system  seems  to  be  able  to  meet  virtually  all  of  their  needs 
at  lower  cost  to  DTIC,  all  users  and  to  the  DoD. 

The  only  disadvantage  of  not  having  a  classified  DROLS  is 
that  a  small  proportion  of  citations  might  be  missing  or  less 
than  complete.  Nevertheless,  the  classified  user  could  order 
the  classified  citations  to  be  delivered  by  mail.  Alternatively, 
if  the  user  were  pressed  for  time,  he  could  order  the  reports 
themselves  with  some  greater  risk  of  ordering  something  that  did 


not  meet  his  requirements.  We  found  that  since  all  authorized 
DTIC  information  would  be  available  to  classified  users  by  mail 
or  other  means,  though  some  small  part  of  it  may  be  delayed  in 
coming  if  DROLS  were  unclassified,  the  problem  boils  down  to  the 
value  of  timeliness  in  obtaining  information,  a  value  that  we 
feel  we  could  not  credibly  determine. 


More  detailed  findings  and  conclusions  are  presented  in 
Chapter  VI. 
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III.  SOME  BACKGROUND  CONSIDERATIONS 


A.  PRIOR  LITERATURE 

There  is  a  considerable  body  of  literature  that  describes 
the  developing  situation  in  information  science;  but  not  all  of 
it  is  relevant  to  the  particular  study  we  have  made  for  DTIC. 
Rather  a  considerable  proportion  is  devoted  to  the  historical 
development  of  the  field  and  some  of  it  to  policy  development . 
This  is  especially  true  of  Ref.  2,  which  deals  with  the  problems 
of  scientific  and  technical  communication  in  both  the  context 
of  the  United  States  and  on  a  worldwide  basis.  Another  treat¬ 
ment  in  this  genre  attempts  to  forecast  the  situation  in  1985 
(Ref.  3)»  Reference  4,  on  the  other  hand,  is  more  focused  on 
the  problems  of  DTIC.  This  study  was  done  in  1975  for  DDC  by 
Auerbach  Associates  and  is  labeled  as  a  10-year  Requirements 
and  Planning  Study.  The  work,  which  is  based  almost  entirely 
on  a  large  survey  of  DDC  users,  emphasizes  the  user's  point  of 
view  in  developing  its  conclusions.  This  study  was  focused  on 
all  DDC  services,  whereas  the  current  study  is  directed  only  to 
DROLS . 

One  finding  of  the  Auerbach  report  was  that  users  prefer 
to  deal  with  their  local  library  rather  than  remote  libraries. 
This  finding  encouraged  DDC  to  develop  DROLS  as  a  service  that 
is  available  to  the  local  librarian  rather  than  directly  to 
the  ultimate  user  at  his  desk  or  bench.  DDC  developed  train¬ 
ing  programs  to  teach  librarians  how  to  use  DROLS.  According 
to  the  Auerbach  report  most  users  are  unaware  of  the  full 
range  of  DDC  services.  This  has  encouraged  DTIC  to  engage  in 
some  low  profile  marketing  of  its  services  with  the  result  that 
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many  librarians  are  now  familiar  with  DTIC  services  even 
though  many  bench-level  researchers  and  managers  could  use 
more  instruction  concerning  what  information  support  is 
available . 

One  of  the  Auerbach  report's  more  significant  conclusions 
was  that  DDC  should  move  to  providing  fact  and  information  ser¬ 
vices  as  well  as  bibliographic  and  document-oriented  services. 
This  same  kind  of  recommendation  appeared  in  our  limited  sur¬ 
vey  of  DTIC  users.  Thus  far  the  holdings  of  only  five  Informa¬ 
tion  Analysis  Centers  (IAC)  can  be  identified  on-line  by  DTIC 
users.  Other  IACs  must  be  contacted  directly  by  the  user  with¬ 
out  going  through  DTIC.  To  change  this,  new  DoD  regulations  are 
needed  that  will  permit  DTIC  to  expand  into  new  data  files.  A 
recent  DTIC  Digest  indicates  that  they  are  making  plans  to  move 
in  this  direction.  In  addition,  the  Shared  Bibliographic  Input 
Experiment  (SBIE)  potentially  will  build  a  Defense  On-Line  Cat¬ 
alog  at  DTIC  which  also  will  serve  as  a  referral  clearinghouse 
for  documents  and  data  files  from  sources  other  than  DTIC  within 
the  Defense  Community. 

B.  THE  AUERBACH  SURVEY  OF  INSTITUTIONS 

Part  of  the  Auerbach  Associates'  study  for  DDC,  the  "DDC 
10-year  Requirements  and  Planning  Study,"  involved  a  survey  of 
fourteen  U.S.  Government  agencies.  This  survey  was  called  the 
Interagency  Survey  Report  (Ref.  5)  and  its  purpose  was  to  de¬ 
velop  a  projection  of  goals,  objectives,  operations,  services, 
and  structure  of  the  information  environment  that  DDC  will  be 
facing  through  1988.  The  findings  were  the  following: 

1.  DDC’s  primary  role  is  to  serve  the  DoD  RDT&E 
community . 

2.  DDC’s  secondary  role  is  to  serve  Federal  agencies. 

3.  DDC's  tertiary  role  is  to  advance  the  state  of 
the  art  of  information  processing  in  the  nation. 


The  study  points  up  the  fact  that  technological  problems 
of  information  processing  are  not  as  significant  in  defining 
the  scope  and  nature  of  services  as  are  the  organizational 
and  economic  ones.  Also,  the  available  technology  has  not  yet 
been  exploited  fully  by  the  information  community.  The  report 
urges  that  the  parochial  interests  of  military  management 
information  programs  should  give  way  to  achieve  implementation 
of  effective  RDT&E  management  information  systems.  They  admit 
that  no  value  measures  of  information  have  been  developed  that 
would  be  useful  in  making  economic  decisions.  Rather,  informa¬ 
tion  has  subjective  and  pragmatic  value  to  its  users,  but 
there  are  no  measures  of  this  value.  This  finding  is  consis¬ 
tent  with  opinions  and  attitudes  we  found  in  our  own  user 
survey.  The  Auerbach  report  further  urged  DDC  to  develop  and 
apply  econometric  measures  to  information  processes  and  to 
increase  its  efforts  to  market  DDC  products  and  services. 

The  Auerbach  study  also  suggests  that  DDC  develop  new 
products  to  satisfy  a  wider  range  of  DoD  information  needs. 

For  example,  an  effective  RDT&E  Management  Information  System 
would  help  RDT&E  managers  assess  their  options  at  decision 
points  in  their  programs. 

C.  THE  AINES  REPORT 

In  January,  1978,  a  report  (Ref.  6)  was  published  by  OUSDRE 
(Research  and  Advanced  Technology).  The  author,  Andrew  A.  Aines, 
was  a  senior  staff  associate  of  the  National  Science  Foundation 
on  loan  to  OUSDRE  (R&AT).  The  purpose  of  the  study  was  to 
appraise  the  health  of  the  DoD  Scientific  and  Technical  Infor¬ 
mation  (STI)  programs.  It  was  intended  to  result  in  improvement 
and  growth  of  these  programs. 

This  study  was  undertaken  in  response  to  the  rapidly 
emerging  large-scale  use  of  advanced  information  technology, 
in  recognition  of  the  need  for  improved  effectiveness  of  the 
many  DoD  information  systems  and  also  in  recognition  of  the 


fact  that  more  than  ten  years  had  passed  since  the  last  formal 
study  of  DoD  STI  programs.  Other  factors  were  the  enormous 
proliferation  of  information  and  recognition  of  the  importance 
of  information  as  a  lever  of  power.  The  study  was  intended  to 
suggest  improvements  that  would  be  useful  and  compatible  with 
how  DoD  STI  programs  should  look  in  the  mid-1980s  time  period. 
The  study  involved  surveys  and  interviews  with  research  and 
engineering  managers  as  well  as  managers  of  DoD  technical 
information  programs. 

Aines  concluded  that  although  the  DoD  STI  program  was  ac¬ 
tive  and  operative  it  had  a  number  of  weaknesses,  particularly 
in  the  area  of  management,  that  needed  to  be  rectified.  The 
management  problems  identified  included  the  following: 

1.  Directives  and  regulations  were  outdated. 

2.  STI  had  low  priorities  in  funding  and  in 
command  attention. 

3.  Reorganizations  were  too  frequent,  personnel 
turnover  was  too  high,  and  the  status  of 
information  managers  was  too  low. 

4.  Interaction  between  R&E  managers  and 
information  managers  was  too  low. 

5.  Data  bases  were  not  being  used  effectively. 

6.  The  use  of  advanced  information  technology 
was  not  moving  evenly  or  smoothly. 

7.  The  benefits  of  a  government -wide  STI  system 
featuring  electronic  interchange  were  not 
receiving  enough  attention. 

Despite  these  management  problems,  DDC  was  probably 
(and  still  is)  the  most  advanced  information  agency  in  the 
federal  government.  However,  it  was  not  using  its  capabilities 
fully  to  aid  in  improving  National  policies  and  programs. 
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Aines  recommended  various  measures  to  offset  these  mana¬ 
gerial  problems.  He  urged  that  long-  and  mid-range  plans  be 
developed  to  relate  OUSDRE,  the  military  services  and  the  other 
Defense  agencies.  A  DoD  coordinating  council  should  be  formed. 

A  task  group  should  be  created  to  revise  outdated  STI  directives 
and  regulations  and  annual  reviews  of  STI  should  be  undertaken 
in  OUSDRE  and  other  Defense  agencies. 

Relationships  between  information  agencies  and  other 
kinds  of  organizations  such  as  professional  societies,  educa¬ 
tional  institutions  and  other  governmental  units  should  be 
studied  and  improved.  Supervision  of  STI  programs  should  be 
improved  and  collaboration  among  the  STI,  logistical  and 
intelligence  communities  should  be  fostered.  DDC  should  be 
upgraded  to  provide  better  support  for  R&D  managers  and  re¬ 
search  workers,  and  as  a  sign  of  this  upgrading,  its  name 
should  be  changed  to  Defense  Technical  Information  Center — 
a  suggestion  that  has  since  be-en  adopted. 


IV.  USER  ATTITUDES  TOWARD  DROLS  CLASSIFICATION 

A.  INTRODUCTION 

Under  the  revised  subtask  b  (see  Appendix  A)  IDA  was 
requested  to  survey  a  sample  of  DROLS  users  to  obtain  their 
reactions  to  the  proposition  that  DROLS  become  a  completely 
unclassified  system.  The  amended  subtask  c  was  also  dealt  with 
by  analyzing  the  survey.  This  chapter  describes  the  survey 
and  presents  the  results  and  conclusions  therefrom. 

B.  METHOD  OF  STUDY 

One  approach  to  this  study  involved  a  questionnaire  (see 
Appendix  C)  which  was  devised  to  be  easy  to  use  and  convenient 
for  both  the  respondent  and  the  surveyor.  Originally  we  had  in¬ 
tended  to  survey  all  users  of  DROLS  (about  100)  but  then  we  were 
informed  that  such  a  survey  would  involve  severe  problems  in  ob¬ 
taining  permission.  However,  we  could  interview  people  at  up  to 
about  ten  sites,  plus  personnel  at  IDA,  without  problems.  So  we 
decided  that  we  would  go  to  eight  major  users  in  the  Washington, 
D.C.,  area  and  two  outside.  This  would  avoid  incurring  heavy 
travel  costs.  Since  it  was  impossible  to  do  a  statistically 
valid  survey  with  only  eleven  respondents,  we  felt  the  best  we 
could  do  was  to  try  to  get  some  insights  about  user  attitudes 
with  this  number  of  respondents.  Nevertheless,  we  did  try  for 
a  distribution  of  types  of  users  to  ensure  that  whatever  opin¬ 
ions  we  did  get  would  be  reasonably  representative.  We  used 
the  following  matrix  (Exhibit  IV-1)  as  a  guide,  and  tried  to 
provide  a  distribution  of  respondents  as  shown  throughout  the 
matrix . 
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EXHIBIT  IV-1 


DISTRIBUTION  OF  SURVEY  RESPONDENTS 
(U  =  Unclassified  Terminal,  C  =  Classified  Terminal) 


LIBRARIES 
&  SCHOOLS 

R&D 

LABORATORY 

OPERATING 

COMMAND 

Army 

Army 

Library  (U)(C] 

BRL  (C) 

Navy 

NSWC  (C) 

NRL  (C) 

NAVMAT  (C) 

Air  Force 

AFOSR  (U) 

AFAL  (C) 

DOD 

DARPA  (U) 

DIA  (C) 

Contractor 

IDA  (C) 

Lincoln  Lab  (C) 

The  method  used  in  setting  up  the  interviews  was  to  tele¬ 
phone  the  individual,  usually  a  librarian  or  information  source  # 

person,  listed  as  the  DTIC  contact  in  a  DTIC  listing  of  termi¬ 
nal  sites.  This  individual  was  told  what  we  were  planning  and 
that  he  or  she  would  shortly  receive  a  packet  containing  some 
descriptive  material  and  several  questionnaires.  After  they  • 

had  time  to  examine  the  material,  they  were  recontacted  and  an 
appointment  for  interviews  was  set  up.  Each  contact  was  asked 
to  line  up  two  other  individuals  for  interviews — one  a  resear¬ 
cher,  the  other  a  research  manager.  We  had  two  reasons  for  this  • 

elaborate  procedure.  First,  we  felt  that  it  would  be  more  pro¬ 
ductive  if  people  had  an  opportunity  to  read  over  the  material 
and  questionnaires  before  responding,  particularly  since  some  of 
the  questions  might  involve  finding  some  data.  Second,  we  felt  • 
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that  although  seemingly  less  efficient  than  the  use  of  a  mailed 
questionnaire,  the  face-to-face  interview  would  enable  us  to 
learn  more  and  to  detect  nuances  of  intended  meaning  because  we 
could  probe  somewhat  further  if  the  answers  to  some  questions 
(or  if  the  questions  themselves)  seemed  inappropriate.  Further¬ 
more,  we  would  be  there  to  explain,  as  necessary,  any  or  all  of 
the  questions. 

As  it  turned  out,  this  mode  of  conducting  the  survey  was 
very  fruitful.  Nearly  all  individuals  we  surveyed  had  read 
the  material  beforehand  and  had  marked  up  their  questionnaires. 
Where  cost  and  other  data  were  requested  they  had  taken  the 
trouble  to  look  up  the  information.  The  interviews  themselves 
generally  went  very  smoothly  and  quickly  since  the  primary  task 
was  to  transfer  information  from  their  questionnaires  to  ours. 

In  most  cases,  the  longest  part  of  the  interview  involved  a  dis¬ 
cussion  of  question  16,  where  the  subjects  were  asked  for  their 
ideas  on  improvements  to  DROLS,  or  involved  a  general  discussion 
of  how  each  individual  used  on-line  information  in  his  work. 

The  questionnaire  and  introductory  material  that  was  sent 
out  are  reproduced  in  Appendix  C.  The  findings  are  based  on 
responses  of  28  individuals  at  11  organizations. 

C.  WHAT  WE  LEARNED 

What  we  learned  from  the  survey  is  not  surprising.  Those 
individuals  at  sites  having  classified  terminals  were  almost 
universal  in  rejecting  the  idea  of  an  unclassified  DROLS,  some 
of  them  emphatically.  Some  said  that  without  being  able  to  get 
classified  information  on-line,  the  whole  point  was  lost  and 
there  would  be  no  reason  for  DROLS  to  continue  existing.  The 
essential  rationales  for  this  attitude  were: 

1.  The  cost  of  a  classified  terminal  is  low  compared  to 
the  value  of  the  information  that  can  be  received  in 


a  timely  manner  from  it.  The  researcher's  time  is  a 
far  more  precious  quantity. 

2.  Many  times  quick  response  tasks  are  requested  of  re¬ 
searchers.  Waiting  for  one  or  two  weeks  for  informa¬ 
tion  at  the  beginning  of  such  a  task  would  jeopardize 
their  ability  to  meet  the  deadline. 

3.  A  number  of  researchers  use  an  interactive  mode,  look¬ 
ing  over  the  shoulder  of  the  terminal  operator  while 

a  search  is  being  done.  Thus,  they  are  able  to  guide 
the  search  more  efficiently  and  ensure  that  it  is 
structured  appropriately. 

On  the  other  hand,  the  possessors  of  unclassified  termi¬ 
nals  were  generally  happy  with  what  they  had  and  were  pleased 
to  learn  that  their  terminals  would  be  capable  of  providing 
even  more  information  to  them  in  the  future.  They  obviously 
did  not  object  to  DROLS  being  unclassified.  In  fact,  due  to 
the  nature  of  their  work,  many  users  are  not  able  to  use  clas¬ 
sified  systems.  The  Army  Library  illustrates  this  point.  It 
has  a  classified  and  an  unclassified  terminal,  both  of  which 
are  in  substantial  use.  The  classified  terminal  is  used  by 
members  of  the  military  staffs  at  the  Pentagon  for  project  and 
program  information.  The  unclassified  terminal  is  used  to  answer 
requests  for  information  under  the  Freedom  of  Information  Act, 
press  and  media  requests  for  information,  and  thesis  information 
for  members  of  the  Armed  Forces.  Since  none  of  the  latter  can 
contain  classified  information,  the  unclassified  terminal 
provides  a  desirable  automatic  screen  that  assures  them  that 
they  can  use  freely  whatever  information  they  receive. 

D.  COMMENTS  ON  OTHER  FACTORS  IN  THE  SURVEY 

Of  our  twenty-eight  respondents,  fifteen,  or  more  than  half, 
were  librarians  or  information  resource  persons,  most  of  whom 
actually  used  the  terminals.  The  reason  for  the  preponderance 


of  persons  of  this  type  was  that  we  had  at  least  one  from  each 
organization  who  was  the  initial  contact.  In  addition,  some¬ 
times  the  contact  was  timid  about  volunteering  information  un¬ 
less  his/her  supervisor  was  present .  In  several  cases  both 
individuals  were  interviewed  simultaneously,  but  even  where 
separate  interviews  were  conducted  the  response  to  any  partic¬ 
ular  question  was  the  same.  This  seemed  to  be  true  at  most 
organizations.  That  is,  where  three  or  four  individuals  were 
interviewed  at  an  organization,  they  all  generally  felt  the 
same  about  DROLS  and  its  need  for  classification.  Hence,  it  is 
not  certain  whether  the  library  or  resource  people  were  simply 
reflecting  the  views  of  the  researchers  they  served  or  whether 
they  had  formed  their  opinions  independently.  At  one  site,  we 
believe  that  permission  to  respond  to  the  survey  had  to  be 
granted  by  an  officer  of  the  institution,  and  that  he  had  given 
instructions  about  how  to  respond  to  the  questions  about  clas¬ 
sification.  All  four  respondents  (two  librarians)  at  this  site 
were  strongly  opposed  to  -an  unclassified  DROLS. 

This  illustrates  one  possible  weakness  of  our  method  of 
polling  where  the  organizational  hierarchy  has  the  opportunity 
to  review  the  questions  and  to  set  out  the  organizational  point 
of  view  before  the  interviews  are  conducted.  On  the  other  hand, 
the  views  of  management  are  given  an  opportunity  to  be  repre¬ 
sented  . 

This  clearly  did  not  happen  at  some  other  places.  One 
site  provided  examples  where  opposite  opinions  were  found  in 
the  same  organization.  One  individual,  a  researcher,  insisted 
that  he  needed  information  quickly  in  order  to  do  his  work.  He 
declared  also  that  if  he  could  not  get  information  right  away 
there  was  a  strong  chance  that  he  would  no  longer  be  interested 
in  it.  At  the  same  site,  a  manager  said  that  although  he  needs 
classified  information  for  his  work,  he  could  be  patient  and 
wait  for  a  DTIC  search  since  he  was  never  in  that  much  of  a 


hurry  for  the  results.  In  fact,  he  much  prefers  the  format  of 
the  DTIC  search  bibliography,  a  neat,  stapled  hard  copy  listing 
rather  than  the  aluminized  paper  roll  that  he  receives  from 
the  local  terminal. 


Nearly  every  organization  we  visited  had  one  or  more  ^ 

unique  problems,  thus  making  it  more  difficult  and  risky  to 
generalize  from  this  small  sample  to  the  entire  possible 
spectrum  of  users.  For  example,  one  DoD  organization,  which 
one  would  think  should  have  a  classified  terminal,  has  only  •  ^ 

an  unclassified  one,  although  they  have  been  trying  to  obtain 
a  classified  one  for  some  time.  The  problem  seems  to  be  some¬ 
thing  as  trivial  as  who  is  to  pay  for  the  installation  of  an 
electrical  wall  socket  in  the  vault  space  designated  for  the  9 

terminal.  In  another  unique  situation,  an  agency  has  what 
seems  to  be  a  ridiculously  limited  budget  for  acquisition  of 
reports,  so  it  insists  on  having  as  complete  information  as 
possible  on  each  report  before ‘it  orders,  since  it  must  be  sure  • 

to  order  only  reports  that  are  needed.  Despite  these  special 
circumstances  we  believe  that  the  results  of  the  survey  identify 
the  major  needs  felt  by  most  DROLS  users.  In  fact,  if  any 
group  is  unrepresented  in  this  survey  it  is  the  non-users  of  # 

DROLS  whose  opinions  were  not  sought;  however,  we  did  have  some 
informal  input  from  other  contacts  outside  of  the  survey  that 
indicated  they  were  insufficiently  aware  of  DROLS  capabilities 
to  make  active  use  of  the  system.  Nevertheless,  they  felt  they  • 

were  getting  good  service  from  DTIC  when  they  requested  a  DTIC 
search  by  mail  or  telephone. 


E.  RESULTS  (See  Exhibits  IV-2  and  IV-3) 


To  an  overwhelming  extent  (21  out  of  27),  DROLS  users  in¬ 
dicated  their  preference  for  a  classified  DROLS  over  an  unclas¬ 
sified  system.  The  usual  rationale  for  this  was  that  their  work 
required  classified  information  and  they  had  to  have  it  reason¬ 
ably  fast  so  that  they  could  complete  their  tasks  in  the 
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EXHIBIT  I V - 2 


WHAT  WE  FOUND 


Answers  to  question 

14  .  .  .  WOULD  YOU 

BE  SATISFIED 

WITH  AN 

UNCLASSIFIED  DROLS? 

Unclassified 

Classified 

Terminal 

Terminal 

Total 

Strong  Yes 

2 

1 

Moderate  Yes 

1 

2 

3  ) 

Indifferent 

- 

1 

l 

Moderate  No 

1 

6 

7  l  21 

Strong  No 

1  (Anomaly) 

13 

14  J 

#  Rationales:  Cl  assified  ,,,  need  classified  information,  need  rapid 

response,  Cost  is  not  a  factor 

Unclassified  , , ,  need  unclassified  information  uncon- 

#  taminated  by  classified  for  FOIA 

REPLIES,  PRESS  AND  ACADEMIC  SEARCHES 
( I , E , ,  THESES  AND  PAPERS),  No  NEED 
FOR  CLASSIFIED  INFORMATION-CAN  WAIT 

#  FOR  DTIC  SEARCH,  ONE  INDIVIDUAL  PREFERS 

FORMAT  OF  DTIC  SEARCH, 
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available  time.  This  attitude  was  firm  despite  our  suggestion 
that  an  unclassified  DROLS  would  be  much  better  in  the  future 
than  it  is  today.  But  the  users  seemed  to  believe  that  they 
would  be  missing  some  citation  that  might  be  important  if  they 
could  not  get  classified  information  on-line.  Their  point  of 
view  was  that  their  time  and  the  importance  of  the  work  they 
were  doing  justified  any  differential  in  costs  between  a  clas¬ 
sified  and  an  unclassified  terminal.  Only  one  respondent  in 
answer  to  question  15  indicated  that  the  cost  of  a  classified 
terminal  is  too  high.  No  other  response  indicated  that  anyone 
thought  that  cost  was  a  factor  in  this  question  (e.g.,  see 
answers  to  question  5).  Moreover,  all  of  those  we  talked  to 
already  had  sunk  costs  in  their  setup.  Removing  the  classified 
system  would  waste  most  of  the  sunk  costs. 

Although  we  must  recognize  the  contrary  views  of  our 
respondents,  there  are  some  strong  arguments  for  going  to  an 
unclassified  DROLS.  For  example,  although  the  users  want 
assurance  that  they  are  getting  complete  bibliographies,  a 
number  of  factors  besides  classification  limit  the  complete¬ 
ness  of  information  provided  by  a  classified  search.  We 
shall  discuss  these  here: 

•  Classification :  By  regulation  DTIC  has  no  Top 
Secret  information  in  its  files.  Neither 

can  it  have  cryptographic  and  communications 
security  or  communications  and  electronic 
intelligence.  Moreover,  the  searcher  and  his 
organization  must  have  a  need-to-know  in  the 
particular  areas  in  which  he  is  searching. 

•  Limited  Distribution:  Roughly  50  percent  of  the 
DTIC  Technical  Report  files  are  limited  in  some 
way  other  than  classification.  Limits  are  placed 
by  the  originating  agency  of  the  study.  They  may 
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permit  distribution  of  the  entire  summary 
to  DoD,  Government,  or  anyone.  In  addition, 
other  limitations  may  be  placed  on  contractors 
depending  on  their  fields  of  interest .  A  dis¬ 
tribution  limitation  in  DROLS  denies  only  the 
abstract  but  not  the  complete  citation  of  a 
limited  document  as  it  would  appear  in  TAB. 

If  a  Technical  Report  is  unclassified  and 
unlimited,  it  is  provided  to  NTIS,  which  then 
makes  it  available  for  public  sale.  This 
happens  in  the  case  of  about  50  percent  of 
all  Technical  Reports.  NTIS  leases  the  bib¬ 
liographic  tapes  of  this  information  to  one 
or  more  of  the  commercial  data  bases. 

Independent  Research  and  Development  File: 

Since  the  IR&D  files  contain  information  that 
is  regarded  as  proprietary,  no  contractors, 
not  even  Federal  Contract  Research  Centers 
(FCRC's),  are  permitted  to  access  these  files 
although  they  are  available  to  DoD  users. 

Data  Not  in  Files:  Despite  clear,  unambiguous 
regulations  and  directives  regarding  the  submis¬ 
sion  of  reports  and  work  unit  data  to  DTIC  (see 
Part  A,  Chapter  II  of  this  report)  estimates  re¬ 
veal  that  about  one-half  of  the  information  that 
should  be  submitted  to  DTIC  is  not  (Refs.  7,  8, 
9).  We  have  little  evidence  of  the  reasons  for 
this,  but  we  infer  that  there  are  two  major  ones 


(1)  the  responsible  individuals  do  not  consider 
the  submission  of  reports  to  DTIC  as  a  serious 
responsibility,  perhaps  because  there  are  no 
sanctions  in  the  regulations  or  directives  if 
they  are  not  obeyed,  and  (2)  there  are  some 
organizations  or  individuals  in  the  Services  who 
do  not  want  to  make  public  (even  to  the  limited 
DoD  public)  what  they  have  been  doing.  This 
suggests  that  DTIC  should  undertake  some  atti- 
tudinal  research  to  determine  how  it  might 
best  obtain  cooperation  from  those  who  are 
withholding  information. 

•  Purging  of  Files:  To  conserve  on-line  storage 
space,  DTIC  removes  all  records  (except  AD  num¬ 
ber)  in  the  interactive  retrieval  files  for 
documents  that  are  more  than  ten  years  old. 

Thus,  a  subject  search  on-line  that  might  turn 
up  older  documents  is  incomplete  and  must  be 
continued  off-line  by  examining  hard-copy  docu¬ 
ments.  The  alternative  is  to  order  an  off-line 
printout  including  a  "delayed  search,"  which  pro¬ 
vides  the  full  records  for  all  the  cited  documents. 

Another  argument  for  making  the  system  unclassified  is 
that  it  would  be  possible  to  interface  with  the  various  informa¬ 
tion  and  computer  resource  sharing  networks  (e.g.,  ARPANET)  that 
can  handle  only  unclassified  material.  This  could  allow  greater 
use  and  flexibility  of  the  system  in  acquiring  and  manipulating 
STI .  Furthermore,  an  unclassified  system  could  be  controlled 
and  managed  by  means  of  existing  software  packages  of  the  type 
now  used  by  the  commercial  data  bases .  This  would  overcome  some 
of  the  more  common  criticisms  of  DROLS  that  we  received  from  the 
users  (see  next  paragraph  (F)  of  this  section). 


We  conclude  that  a  searcher  in  DTIC ' s  files  is  usually 
going  to  receive  only  a  large  fraction  (perhaps  no  more  than 
half)  of  all  the  information  that  he  should  and  would  like 
to  have  no  matter  how  efficient  and  skilled  a  searcher  he  is. 
This  is  true  whether  he  has  a  classified  or  unclassified  termi¬ 
nal.  Using  a  classified  terminal  may  result  in  receiving  at 
most  a  few  more  citations  than  in  the  average  search  over  an 
unclassified  terminal,  but  it  cannot  ensure  completeness  until 
the  other  information  flow  inhibitors  are  overcome. 

So  we  conclude  that  a  classified  system  may  provide  more 
information  than  an  unclassified  one,  although  at  this  point 
we  do  not  have  the  data  that  tells  how  much  more  complete  or 
useful  for  research  and  development  such  a  system  would  be. 

Despite  these  positive  reasons  for  converting  DROLS  to  an 
unclassified  system  there  are  some  strong  reasons  for  not  taking 
this  action.  First  of  all,  most  classified  users  are  strongly 
opposed,  because  they  feel  that  they  would  be  denied  significant 
information  if  they  could  not  access  classified  files.  Statis¬ 
tical  arguments  (about  the  proportion  of  classified  information 
in  the  files)  do  not  sway  this  point  of  view.  The  counter¬ 
argument  is  made  by  users  that  some  areas  of  technology  are 
nearly  all  classified  and  an  unclassified  search  in  such  an 
area  might  result  in  few  if  any  of  the  possible  finds.  They 
claim  that  it  does  not  matter  that  there  are  many  stronger  inhib 
itors  than  security  classification  (see  above),  to  the  flow  of 
information,  the  fact  is  that  they  have  the  need  (and  right)  to 
obtain  such  information  through  a  real-time  system  such  as  DROLS 

It  is  difficult  to  argue  with  this  point  of  view:  these 
users  have  in  fact  paid  their  dues  by  installing  the  proper 
facilities  for  receiving  classified  information  and  by  having 
secure  environments  for  the  use  and  storage  of  such  informa¬ 
tion.  They  see  no  reason  why  it  should  be  denied  to  them  by 
what  might  appear  to  be  an  arbitrary  decision  by  DTIC. 


F.  SOME  OTHER  COMMENTS  ON  DROLS 


All  users  were  complimentary  about  DROLS  and  indicated 
that  they  were  pleased  with  the  service.  Nevertheless,  many 
of  them  had  comments  on  how  they  would  like  to  see  the  service 
improved.  Nearly  all  of  these  comments  came  from  the  libra¬ 
rians  or  information  resource  persons  who  actually  used  the 
terminals.  Although  these  comments  are  not  new  to  DTIC,  we 
believe  that  it  would  be  of  interest  here  to  indicate  to 
DTIC  what  some  of  the  concerns  of  the  users  are  in  the  context 
of  this  study. 

The  gist  of  most  of  the  comments  we  received  was  that 
they  would  like  to  have  DROLS  operate  in  the  same  way  that 
Lockheed’s  DIALOGUE  and  other  commercial  on-line  services  now 
operate.  For  example: 

•  DTIC  should  provide  a  full  Boolean  search 
strategy.  The  partial  Boolean  now  available 

is  too  limiting  for  sophisticated  and  efficient 
searching . 

•  An  on-line  thesaurus  would  be  helpful. 

•  Also  helpful  would  be  the  ability  to  narrow 
a  search  without  starting  over  again. 

•  A  higher  response  speed  on  page  shifting  would 
also  be  desirable. 

•  There  is  no  convenient  way  of  tracking  from 
work  units  to  the  technical  reports  that 
resulted  and  vice  versa. 

•  Many  documents  such  as  proceedings  of  symposia 
and  meetings  and  classified  journals  have  indi¬ 
vidual  articles  in  them  that  are  not  reachable 
as  citations  in  DROLS  unless  they  have  been 
entered  in  the  files  separately.  This  results 


in  loss  of  availability  of  many  possibly  valuable 
citations.  The  suggestion  is  to  provide  indivi¬ 
dual  cataloging  for  the  separate  papers  included 
in  such  documents. 


•  The  terminology  in  DRIT  is  years  behind  the 
current  terms  in  such  fields  as  electronic 
£  materials.  This  inhibits  the  ability  of 

searchers  in  advanced  fields  to  find  citations. 
What  the  librarian  would  like  is  a  more  flexi¬ 
ble  system  that  would  enable  the  users  to 
q  search  on  their  own  terms  which  presumably 

represent  the  most  advanced  thinking  in  each 
field  rather  than  being  restricted  to  a  dic¬ 
tionary  of  obsolete  terms.  Limited  text 
•  searching  capability  has  been  introduced  in 

DROLS  recently,  but  it  does  not  fully  meet 
this  need. 


•  •  Some  DROLS  users  also  put  program  planning, 

work  unit  and  technical  report  data  into  the 
system.  It  would  be  helpful  to  such  users  to 
have  an  on-line  format  generator. 

• 

•  For  documents  older  than  ten  years  only  an  AD 
number  is  available  on-line.  Since  ten  years 
is  an  arbitrary  length  of  time  to  decree  that 
•  work  in  technology  is  obsolete  it  would  be 

desirable  to  provide  some  way  of  getting  more 
information  about  such  documents  on-line. 

®  •  Many  respondents  are  concerned  about  the  time 

consumed  by  mailed  reports  and  bibliographies 
getting  to  their,  requestors  even  though  they 
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are  in  the  local  DTIC  area.  We  heard  of  delays 
approaching  two  weeks,  although  ten  days  is 
more  frequent  and  six  days  seems  to  be  a  mini¬ 
mum.  It  is  difficult  to  apportion  these  times 
among  DTIC,  the  U.S.  Postal  Service  and  internal 
processing  and  distribution  at  the  requestor's 
organization.  DTIC  claims  to  get  reports  and 
requested  bibliographies  into  the  mail  in  4-6 
days.  This  could  be  an  upper  limit  to  delivery 
time  if  DTIC  used  a  local  courier  service  to 
deliver  documents  to  Washington  area  users  on 
the  day  they  were  ready. 


EXHIBIT  I V - 3 


SURVEY  SCORE  BOARD 


1 . 

Please  identify  yourself: 

N 

% 

• 

a . 

Information  Specialist 

1  5 

54 

b. 

Researcher 

8 

29 

c . 

Research  Manager 

_5 

18 

• 

TOTALS 

28 

100 

2. 

Estimate  your  use  of  DROLS  over  the 

last 

year. 

a . 

Daily 

11 

39 

• 

b. 

Weekly 

20 

c . 

Monthly 

10 

36 

d. 

Yearly 

l!s(1) 

5 

e . 

Never 

0 

0 

• 

(See  Note  1)  TOTALS 

28 

100 

3. 

What 

proportion  of  your  collection 

0  - 

9  0% 

of  documents  is  classified? 

• 

4. 

Have 

i  you  used  commercial  on-line 

bibliographic  services? 

a . 

Yes 

16 

57 

b. 

No 

7 

25 

• 

c . 

No  Response 

5 

18 

TOTALS 

28 

100 

5. 

Why 

do  you  not  have  a  classified  terminal 

? 

£ 

a . 

No  need  for  classified  information 

1 

17 

b. 

Little  need  for  classified 

i nf ormat i on 

3 

50 

c . 

Cost 

0 

0 

d. 

Terminal  procurement  difficulties 

0 

0 

• 

e . 

Secure  facility  difficulties 

4 

67 

f. 

Information  otherwise  available 

2 

33 

g  • 

Other 

L 

17 

TOTALS 

10(6) (3) 

184^ 

31 


% 


10 


12 


If  only  unclassified  service  were 
provided  what  would  you  do  about 
missing  information? 


expense  of  a  classified  terminal? 

a.  Need  classified  information 

1  9 

83 

b.  Response  speed 

14 

61 

c.  Cost  not  a  problem 

3 

13 

d.  Terminal  provisions  not  a  problem 

0 

0 

e.  Other 

0 

0 

TOTALS 

36(23) ^ 

1  57 

Would  you  consider  changing  to  an 
unclassified  terminal? 

a .  Yes 

3 

13 

b.  No 

17 

74 

c.  No  Response 

3 

13 

TOTALS 

23 

100 

How  was  the  capital  expense  of  your 
terminal  funded? 

a.  Library 

8 

29 

b.  Organization 

7 

25 

c.  Project 

2 

7 

d.  Other  or  no  response 

14 

50 

TOTALS 

32  (  28  )^ 

111 

Who  funds  the  operational  expense  of 
your  terminal? 

a.  Library 

7 

25 

b.  Organization 

7 

25 

c.  Project 

2 

7 

d.  Other  or  no  response 

14 

50 

TOTALS 

30(28)(3) 

107 

‘(2) 


‘(2) 


'(2) 


32 


a . 

Nothing 

5 

22 

b. 

Order  classified 

entries 

11 

48 

c . 

Order  classified 

reports 

1  3 

57 

d. 

No  response 

3 

1  3 

TOTALS 

32(23)(3) 

112 

14.  Would  you  be  satisfied  with  an 

unclassified  DROLS?  (See  summary 
of  this  question  in  Exhibit  I V - 2 ) 
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Why? 


a . 

No  strong  need  for  classified 
bibliography 

3 

1 1 

b. 

Can  function  adequately  without 

4 

14 

c . 

Cost  of  classified  terminal  too 
high 

1 

4 

d. 

Can  use  DTIC  searches 

5 

18 

e . 

Difficult  to  provide  for  classified 
termi nal 

0 

0 

f . 

Strong  need  for  classified 
i nf ormati on 

1  5 

54 

g. 

Classified  search  is  faster 

16 

57 

h . 

Search  more  fruitful  in-house 

1  4 

( 3  )  50 
(28)v  208 

(See  Note  2)  TOTALS 

CO 

LD 

NOTES:  (1)  One-half  is  used  to  designate  the  situation  where 

a  user  was  uncertain  between  two  intervals  of  the 
time  scale.  Answer  " e"  is  as  expected  since  the 
respondents  were  selected  as  users. 

(2)  Totals  greater  than  100%  indicate  that  one  or  more 
respondents  gave  multiple  answers  to  the  question. 


(3)  Number  in  (  )  is  actual  number  of  respondents  to 
question . 


V.  DISCUSSION 


A.  GENERAL 

As  we  have  noted  earlier,  DTIC  is  considering  changes  in 
its  on-line  system  that  would  extend  its  responsiveness  through 
making  it  possible  to  use  value-added  communications  networks, 
and  that  would  make  possible  the  use  of  commercial  software  for 
the  operating  system.  The  barrier  to  potential  improvements  is 
the  problem  of  security  classification.  If  DROLS  could  be  un¬ 
classified,  then  more  operating  economies  and  improvements  could 
be  made,  but  if  DROLS  must  remain  classified  then  the  system 
may  have  been  improved  about  as  much  as  it  can  be . 

B.  DROLS  FUNCTIONS 

DROLS  presently  performs  two  functions.  One  of  these  is 
to  provide  bibliographic  services  to  individuals  and  organiza¬ 
tions  engaged  in  research  and  development.  The  second  is  to 
provide  management  information  to  program  managers  and  manage¬ 
ment  offices.  Even  in  this  second  function,  DROLS  is  severely 
limited  in  that  it  does  not  provide  precise  management  informa¬ 
tion.  The  Work  Unit  files  and  the  Program  Planning  files,  for 
example,  tell  who  is  doing  what  and  where  it  is  being  done. 

They  offer  only  order-of-magnitude  cost  information;  for  more 
accurate  information  it  is  essential  to  contact  the  agency 
that  is  actually  doing  or  sponsoring  the  work.  The  Program 
Planning  and  Work  Unit  files  are  no  better  than  a  bibliography; 
they  tell  where  to  find  the  real  information  that  one  might  be 
looking  for. 


It  is  clearly  desirable  that  the  DTIC  system  provide  more 
direct  on-line  access  to  and  manipulation  of  actual  R&D  program 
data  rather  than  bibliographic-type  reference.  The  problem 
here  is  that  security  restrictions  made  such  changes  very  diffi¬ 
cult  if  not  impossible.  Due  to  these  restrictions  the  system 
is  inflexible,  which  means  that  data  bases  cannot  be  added  with¬ 
out  inordinate  difficulty  to  ensure  that  they  are  properly  pro¬ 
tected.  It  is  not  possible  to  program  on-line,  nor  is  it  possible 
for  DTIC  to  use  commercial  program  packages  such  as  those  used  by 
Lockheed  and  SDC  to  identify  and  verify  users  and  to  operate  the 
system.  Not  being  able  to  use  such  packages  presents  a  major 
disadvantage  to  DTIC,  which  is  forced  to  do  all  its  programming 
internally,  and  which  is  saddled  with  an  inflexible  system  that 
is  difficult  to  improve  or  expand. 

The  task  statement  mentions  a  dichotomy  that  exists 
between  the  DTIC  mission  to  disseminate  information  and  the 
requirements  for  security  of  the  classified  information  that 
is  included  in  the  files.  The  authors  assert  that  no  real 
dichotomy  exists.  Rather,  the  primary  function  of  DTIC  is  to 
store  and  disseminate  information  to  all  those  who  need  it. 
Security  regulations  only  limit  dissemination  by  defining  who 
needs  the  information  and  by  prescribing  the  means  to  be  used 
to  prevent  others  from  obtaining  it. 

C.  ALTERNATIVES  FOR  DTIC 

We  discuss  four  courses  of  action  for  DTIC  with  respect  to 
dealing  with  the  on-line  situation  discussed  above. 

1.  Continue  DROLS  as  the  combination  classified-unclas¬ 
sified  system  that  it  currently  is.  Recent  modifica¬ 
tions  of  this  system  enable  DTIC  to  provide  uncleared 
users  with  almost  the  same  service  that  cleared  users 
receive.  .Further  planned  changes  will  make  this  dif¬ 
ference  even  smaller  in  the  future. 


2a.  Change  DROLS  to  a  completely  unclassified  system. 

This  will  simplify  making  the  kinds  of  improvements 
that  many  users  would  like  to  see,  by  making  it  pos¬ 
sible  for  DTIC  to  purchase  software  packages  that 
security  requirements  will  not  permit  and  will  reduce 
operating  costs  for  both  DTIC  and  its  users.  But  this 
also  reduces  the  ways  available  to  cleared  users  to 
obtain  classified  information. 

2b.  Completely  unclassify  the  external  portion  of  DROLS 
but  retain  the  internal  classification  so  that  clas¬ 
sified  searches  can  be  done  in-house.  This  would  re¬ 
duce  costs  to  users  while  increasing  their  inconven¬ 
ience.  Classified  bibliographies  could  still  be 
obtained  by  ordering  a  DTIC  search.  This  mode  would 
save  DoD  and  DTIC  some  money  because  the  DTIC  fac¬ 
ilities  in  Los  Angeles  and  Boston  would  no  longer  be 
needed.  Also,  there'would  be  savings  in  communications 
costs  and  in  the  maintenance  of  crypto  equipment  that 
would  accrue  ultimately  to  the  DoD.  However,  the 
savings  and  convenience  to  DTIC  will  be  limited  be¬ 
cause  it  must  retain  computer  security  for  its  in- 
house  terminals . 

3.  An  intermediate  option  between  these  alternatives  is 
to  create  two  essentially  separate  systems,  one  clas¬ 
sified,  the  other  unclassified,  using  two  computers 
and  two  separate  and  different  modes  of  communication. 
Then  it  would  be  possible  for  DTIC  to  improve  the 
unclassified  files  in  any  way  that  it  wanted  without 
being  concerned  about  security.  This  might  also 
simplify  the  problem  of  making  improvements  to  the 
classified  system. 

Alternative  3  presents  some  problems  of  its  own.  For 
example,  the  cleared  user,  in  order  to  do  a  complete  search. 
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would  have  to  search  two  data  bases  or  at  best  initiate  the 
search  differently.  Furthermore,  it  is  not  clear  how  the  in¬ 
formation  should  be  split  between  the  bases.  One  way  of  di¬ 
viding  up  the  files  is  by  fields,  placing  all  classified  fields 
in  the  classified  computer  and  everything  else  in  the  unclas¬ 
sified  computer  file.  With  such  a  scheme  the  uncleared  user 
would  be  served  about  as  well  as  today,  while  the  cleared  user 
would  have  to  access  two  files.  This  also  creates  an  input 
problem  for  DTIC  since  the  division  of  the  files  for  this  pur¬ 
pose  would  have  to  be  accomplished  carefully  to  ensure  that 
nothing  classified  was  stored  in  the  unclassified  computer. 

Since  this  alternative  creates  problems  for  both  DTIC  and  its 
classified  users,  we  believe  it  to  be  the  least  desirable  al¬ 
ternative,  especially  considering  how  effective  the  unclassified 
DROLS  has  become.  Moreover,  the  present  system  handles  reliably 
differences  in  classification  from  Unclassified  to  Secret  as 
well  as  need  to  know.  On  top  of  this  it  also  establishes  effec¬ 
tive  control  over  limited  distribution  information 

Also,  we  think  it  inadvisable  to  adopt  alternative  2a  or 
2b  at  this  time.  The  reason  for  this  is  politic  and  psychological 
rather  than  logical  or  economic.  Our  survey  of  users  (see 
Chapter  IV)  shows  that  they  would  object  strenuously  to  changing 
the  system  in  this  way,  even  though  they  are  not  likely  to 
notice  any  essential  difference  in  the  way  that  it  operates. 

But,  rightly  or  wrongly,  they  do  feel  that  they  would  be  denied 
some  vital  information  if  they  could  not  obtain  it  through  their 
terminals,  and  they  would  resent  the  delay  involved  in  trying 
to  obtain  references  by  other  means.  One  aspect  of  this  is 
that  the  cleared  users  seem  to  consider  themselves  as  belonging 
to  some  kind  of  exclusive  club,  to  which  they  have,  in  some 
sense,  paid  their  dues  by  having  a  classified  terminal  and  fa¬ 
cility  and  they  do  not  want  to  lose  any  of  the  benefits  of  mem¬ 
bership,  even  though  such  benefits  may  be  largely  illusory. 
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There  may  be  ways  to  overcome  this  kind  of  objection  to 
an  unclassified  system.  For  example,  even  though  a  cleared 
terminal  could  not  receive  classified  information  in  an  unclass¬ 
ified  system,  it  would  be  given  the  unclassified  fields  as 
unclassified  terminals  now  receive.  If  any  classified  inform¬ 
ation  is  denied  to  a  formerly  classified  terminal  because  the 
system  has  been  made  unclassified,  then  the  document  itself 
could  be  automatically  sent  to  the  requestor  (perhaps  without 
charge)  thereby  demonstrating  a  "membership  benefit." 

D.  ALTERNATIVES  FOR  USERS 

Becoming  unclassified  does  reduce  the  utility  of  DROLS  to 
its  users  who  now  have  classified  terminals.  But,  in  fact,  the 
unclassified  DROLS  has  been  improved  so  much  that  the  classified 
user  will  receive  little  more  information  than  will  the  unclas¬ 
sified  user.  Moreover,  making  the  DROLS  unclassified  does  not 
deny  the  user  the  classified  information  that  he  needs  and  would 
like  to  have.  To  get  such  information  if  he  has  only  an  unclas¬ 
sified  terminal,  the  classified  user  must  request  DTIC  to  do  a 
batch  search  and  to  mail  the  results  to  him.  Thus,  there  is  a 
delay  of  perhaps  six  to  fourteen  days  while  the  user  waits  for 
the  results  of  his  search.  (We  presume  here  that  DTIC  retains 
the  ability  to  do  such  searches  in-house.)  There  is  still  an¬ 
other  alternative  available  to  the  user  who  may  be  in  a  hurry. 

He  can  simply  order  any  classified  documents  that  his  search 
reveals  without  waiting  for  the  complete  citation.  In  this  case 
he  may  receive  some  documents  that  may  not  be  useful  to  him, 
but  he  can  receive  those  documents  as  quickly  as  though  he  might 
have  seen  the  complete  citation.  In  addition,  users  who  are 
knowledgeable  about  the  technical  fields  they  are  working  in 
know  other  sources  of  the  information  they  are  seeking  and  do 
not  need  to  rely  on  DTIC  for  all  of  their  information. 
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So,  in  effect,  what  we  are  dealing  with  primarily  is  some 
extra  time  to  obtain  partial  bibliographical  information.  In 
fact,  this  extra  time  need  not  be  spent  if  the  user  is  willing 
to  order  reports  from  less  than  complete  citations,  and  take 
the  chance  that  some  of  those  reports  that  he  orders  in  this 
way  may  not  be  useful.  If  we  try  to  examine  the  value  of  time, 
here  is  what  we  run  into. 

E.  VALUE  OF  TIME 

The  question  of  the  value  of  time  and  information  is  tricky. 
It  might  seem  possible  to  assess  the  cost  of  a  delay  in  obtaining 
information  by  adding  up  the  cost  of  additional  researchers ' 
time  spent  in  waiting.  But  this  assumes  that  the  researcher 
is  nonproductive  while  waiting,  when  in  fact  he  or  she  could  be 
working  on  another  project  or  studying  unclassified  literature 
until  the  classified  material  arrives.  Another  way  of  looking 
at  the  value  of  delayed  information  would  be  to  examine  the  cost 
to  an  organization  of  not  winning  a  contract  because  a  proposal 
was  late  or  incomplete  due  to  missing  information.  But  this 
presumes  that  the  quality  of  the  proposal  relative  to  other 
proposals  remains  the  same  except  for  the  contribution  of  the 
new  information.  Still  another  way  of  assessing  the  value  of 
delayed  information  would  be  to  determine  the  cost  of  duplication 
of  an  R&D  effort  because  information  was  not  available  in  a 
timely  way. 

With  any  of  these  methods  of  assessing  the  value  of  in¬ 
formation,  there  is  a  presumption  that  the  researcher  and  re¬ 
search  manager  are  not  knowledgeable  about  what  is  happening 
in  their  own  fields  and  that  they  do  not  have  the  resources  or 
resourcefulness  to  use  other  means  of  getting  timely  information. 
Included  among  these  other  means  are  personal  contact  with  in¬ 
dividuals  and  organizations  in  their  own  areas  of  expertise. 

Still  another  way  of  getting  information  can  be  inferred  from 


the  fact  that  DROLS  provides  only  bibliographic  information. 

If  "real"  data  and  facts  are  needed,  the  only  way  that  DROLS  can 
help  is  to  tell  the  user  where  to  find  them,  including  knowledge¬ 
able  individuals,  if  requested.  Citations  to  classified  reports 
will  be  given  to  all  users,  classified  and  unclassified.  However, 
the  classified  citation  is  likely  to  be  somewhat  more  complete 
than  the  unclassified.  Hence,  the  user  receiving  unclassified 
information  will  have  less  information  to  decide  whether  or  not 
the  report  will  be  useful.  If  he  is  uncertain  and  if  it  is  im¬ 
portant  enough  he  can  order  the  report  and  receive  it  after  the 
same  delay  that  the  cleared  user  might  undergo. 

Reference  10  reports  on  a  client  survey  conducted  by  the 
NASA  Industrial  Application  Center  at  the  University  of  Southern 
California.  Telephone  interviews  were  conducted  on  a  random 
sample  of  clients.  Of  the  total  159  clients  interviewed,  more 
than  53  percent  identified  dollar  benefits  of  on-line  searching. 
Furthermore,  a  direct  relationship  was  shown  between  client 
dollars  invested  and  benefits  derived  from  the  research.  The 
average  ratio  of  benefits  to  investment  was  2.9.  We  note  that 
this  ratio  is  a  result  of  the  perception  of  benefits  as  deter¬ 
mined  by  user  and  that  in  some  cases  the  ratio  was  considerably 
higher  (about  10-20).  While  we  would  question  this  method  of 
gathering  data  on  this  subject,  there  seems  to  be  no  question 
in  most  client's  minds  that  they  benefited  significantly  from 
the  information  service. 

Thus,  we  conclude  that  determination  of  the  value  of  infor¬ 
mation  would  be  extremely  difficult  with  results  that  are  not 
likely  to  be  credible  because  they  involve  unacceptable  assump¬ 
tions.  Moreover,  the  denial  of  information  by  an  unclassified 
DROLS  is  a  denial  only  of  partial  bibliographic  information  and 
is  in  no  sense  a  denial  of  the  "real"  information  that  the  user 
would  like  to  have.  It  only  impinges  on  the  decision  as  to 
whether  or  not  to  order  a  report  or  reports  that  the  biblio¬ 
graphic  record  indicates  might  be  useful.  Considering  the 
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general  incompleteness  of  the  bibliographic  record  (see  Para¬ 
graph  IV-E),  we  believe  that  the  classified  DROLS  offers  very- 
little  more  to  the  user  than  would  a  completely  unclassified 
DROLS. 


VI.  FINDINGS  AND  CONCLUSIONS 


* 
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A.  FINDINGS 

1 .  DROLS  Service 

DROLS  serves  a  growing  body  of  users  from  about  100  remote 
terminal  sites.  The  system  provides  both  classified  and  unclas¬ 
sified  bibliographic  and  other  services  to  a  variety  of  users, 
most  of  them  DoD,  or  affiliated  with  DOD.  The  service  is  heav¬ 
ily  used  by  a  few  users  and  only  moderately  used  by  most,  but 
the  instances  of  dissatisfaction  that  we  found  were  related  not 
to  the  basic  service,  but  to  the  fact  that  commercial  on-line 
services  (unclassified)  have  surpassed  DROLS  in  their  ability 
to  provide  ease  of  access  to  and  use  of  a  data  base.  Most  such 
comments  were  variations  of  "Why  can't  DROLS  do  it  like  _ ?" 

2 .  Limitations  on  Effectiveness 

Searching  for  information  in  the  DROLS  files  is  severely 
hampered  by  the  fact  that  (1)  the  files  are  incomplete,  thus 
reducing  citations  available  on-line,  (2)  information  is  not 
in  the  file  because  it  has  a  higher  classification  than  DTIC 
is  authorized  to  receive  or  because  it  is  not  submitted  to  DTIC 
despite  the  fact  that  regulations  require  such  submission.  The 
net  result  of  these  omissions  is  that  a  searcher  can  receive  no 
more  than  about  one-half  of  the  references  that  he  would  like 
and  should  receive  for  completeness.  Since  only  about  seven 
percent  of  the  citations  in  the  TR  file  are  classified,  the 
effect  of  classification  on  denied  information  is  likely  to  be 
a  small  component  in  these  proportions. 
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3.  Classified  versus  Unclassified 


A  survey  of  a  sample  of  users  indicated  that  those  who  now 
have  classified  service  would  be  very  reluctant  to  trade  that 
off  for  a  greatly  improved  unclassified  service  that  would  pro¬ 
vide  nearly  all  the  bibliographic  information  that  the  classified  ^ 

system  includes. 

4.  Costs  (see  Appendix  B) 


For  a  representative  site,  the  one-time  costs  of  installing  a 
classified  terminal  are  about  $11,000,  while  an  unclassified  ter¬ 
minal  could  be  obtained  for  an  initial  one-time  installation  cost 
of  less  than  $100.  We  assume  both  kinds  of  terminals  are  leased. 
Based  on  actual  IDA  experience,  monthly  operating  costs  for  the 
classified  terminal  would  be  about  $1,000  while  the  same  for  an 
unclassified  terminal  might  be  $145  plus  telephone  line  costs. 
Telephone  line  costs  can  be  a  large  variable  depending  on  the 
location  of  the  user’s  facility  and  how  much  connect  time  is  used. 
It  may  be  more  economical  for  some  non-local  users  to  have  a  ded¬ 
icated  telephone  line  even  though  they  have  an  unclassified  ter¬ 
minal  and  could  use  a  dial-up  access  mode.  Since  we  did  not 
interview  people  who  have  no  terminal,  we  have  no  notion  of 
whether  such  costs  deter  organizations  or  individuals  from  ac¬ 
quiring  a  terminal  or  whether  the  difference  between  the  two 
would  influence  the  selection  of  terminal  classification.  But 
among  the  terminal  users  whom  we  did  interview,  the  universal  at¬ 
titude  was  that  the  information  and  the  convenience  in  obtaining 
classified  or  '’all”  information  are  well  worth  whatever  it  costs; 
on  the  other  hand,  these  individuals  do  not  pay  for  these  serv¬ 
ices.  Their  organizations,  and  ultimately  the  DoD,  do. 

DTIC  in-house  costs  may  not  be  significantly  reduced  by 
going  to  an  unclassified  DROLS  unless  DTIC  also  gives  up  doing 
classified  searches.  If  classified  searches  are  provided,  then 
the  entire  internal  operating  system  must  remain  classified  and 
security  arrangements  at  the  Cameron  Station  facility  would  not 
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be  significantly  changed  by  having  an  unclassified  DHOLS .  How¬ 
ever,  savings  could  be  realized  by  closing  down  the  classified 
Boston  and  Los  Angeles  offices,  which  would  no  longer  be  needed. 

Another  change,  which  we  discussed  earlier,  would  be  to 
institute  a  dual  system  with  separate  classified  and  unclassified 
parts.  Here  again  it  is  difficult  to  see  much  savings  from  such 
a  system  with  the  possible  exception  of  using  commercial  software 
for  changes  in  the  unclassified  system.  But  since  the  classified 
services  would  continue  to  be  offered,  such  cost  savings  could 
be  only  minimal. 

DTIC  might  also  make  DROLS  completely  unclassified, 
internally  as  well  as  externally.  Then  all  security  costs  assoc¬ 
iated  with  DROLS  could  be  eliminated,  but  the  savings  would  be 
limited  because  DTIC  itself  must  remain  a  classified  facility 
since  it  must  store  and  supply  classified  documents  for  all 
DoD  agencies.  Other  savings  would  accrue  to  both  DTIC  and  the 
DoD  from  eliminating  the  need  for  maintenance  of  cryptographic 
equipment  and  from  users  optimizing  communications  charges  by 
using  dedicated  lines  or  dial-up,  depending  on  their  volume  of 
business  and  distance  from  Cameron  Station.  Many  local  users, 
for  example,  who  now  pay  for  dedicated  lines,  would  have  virtu¬ 
ally  no  communications  costs  if  DROLS  became  unclassified. 

5 .  Possible  Improvements 

DTIC  clearly  desires  to  improve  its  services  to  its  clients. 
The  clients  would  welcome  better  service  and  some  have  suggested 
what  they  would  like.  The  most  common  suggestion  was  that  DTIC 
should  modify  its  software  to  permit  searching  in  a  similar  way 
to  that  permitted  by  the  commercial  on-line  services,  such  as 
Lockheed  or  System  Development  Corporation  (SDC).  This  problem 
appears  to  stem  from  DTIC  using  "home-grown"  software  rather 
than  commercially  available  programs.  Unfortunately,  security 
requirements  prevent  the  use  of  commercial  software.  These 
improvements  include  complete  Boolean  search  strategy,  on-line 


45 


indexing  and  retrieval  terms,  and  elimination  of  look-up  codes 
for  organizations  and  subject  areas.  We  are  aware  that  these 
suggestions  have  been  made  before,  but  we  feel  that  it  may  be 
useful  to  repeat  them  in  the  context  of  this  report.  A  more 
complete  list  of  user  suggestions  can  be  found  in  Chapter  IV. 

B.  CONCLUSIONS 

•  There  is  no  real  dichotomy,  as  mentioned  in  the 
task  statement  for  this  study,  between  the  services 
that  DTIC  is  designed  to  offer  and  security 
requirements.  Security  requirements  only  state 
limits  as  to  who  is  to  receive  certain  kinds  of 
sensitive  information,  whereas  the  mission  of 

DTIC  is  to  disseminate  information  as  widely  as 
possible,  within  security  limits.  In  fact, 
security  limitations  are  only  one  of  the  lesser 
limiters  of  DTIC  information  disseminated  with 
the  aid  of  DROLS. 

•  The  most  important  limiting  factor  on  DROLS 
effectiveness  is  that  many  documents  are  not 
submitted  to  DTIC  and  hence  not  represented 
in  the  files.  About  50  percent  of  all  docu¬ 
ments  that  should  be  submitted  are  missing 
despite  regulations  and  directives  that  require 
such  submission.  Still  another  important  limit¬ 
ing  factor  is  distribution  limitations  that 

are  frequently  used  to  limit  the  flow  of  infor¬ 
mation  . 

•  The  unclassified  service  that  will  be  offered 
on  DROLS  in  the  next  few  months  will  be  vir¬ 
tually  as  complete  as  the  classified  service. 

•  The  order  of'  overall  desirability  of  alterna¬ 
tives  for  the  future  of  DROLS  is  the  following: 

46 


* 


1. 


m 


Retain  the  present  classified  and  unclassified 
system. 

2.  Make  DROLS  service  to  external  terminals 
completely  unclassified,  but  retain  in- 
house  classified  service. 

3.  Make  DROLS  completely  unclassified,  including 
in-house  service. 

4.  Convert  the  present  DROLS  into  two  separate 
systems,  one  classified,  the  other  unclassified 

•  DTIC  users  have  suggested  ways  to  improve  DROLS 
that  seem  desirable.  Most  common  among  these  is 
the  suggestion  to  design  protocols  for  addressing 
and  using  the  system  similar  to  the  commercial 
on-line  data  bases. 

•  Finding  ways  to  increase  the  completeness  of  its 
files  and  creating  a  more  user-oriented  on-line 
system  would  be  a  researchable  and  desirable 
activity  for  DTIC  to  engage  in. 

•  Users  who  currently  have  classified  terminals 
will  object  strenuously  to  changing  to  an  un¬ 
classified  DROLS.  To  offset  such  objections,  it 
is  suggested  that  such  users  would  be  given 
certain  privileges ,  that  would  not  be  given  to 
unclassified  users,  such  as  automatic  sending 

of  any  classified  document  cited  either  wholly 
or  partially  during  a  DROLS  search  by  a  classified 
user . 
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SUMMARY  OF  RESEARCH  PERFORMED 

In  December  1979,  the  Defense  Technical  Information  Center 
(DTIC)  requested  the  Institute  for  Defense  Analyses  (IDA)  to 
undertake  a  study  of  its  information  retrieval  system  known  as 
the  Defense  Research,  Development,  Test  and  Evaluation  (RDT&E) 
On-Line  System  (DROLS).  The  original  work  statement  involved 
five  subtasks.  Subsequently,  after  some  work  had  been  done  on 
each  of  the  subtasks,  DTIC  announced  some  changes  in  DROLS  that 
had  not  been  scheduled  when  the  original  work  statement  was 
drafted.  Accordingly,  the  study  task  was  redirected  by  mutual 
agreement  to  reflect  the  later  status  of  DROLS  and  the  work 
proceeded.  The  redirection  changed  three  of  the  five  original 
task  statements,  one  was  deleted  and  one  was  unchanged.  In 
this  summary  we  shall  review  the  task  statements,  both  original 
and  redirected,  and  indicate  briefly  what  was  done  about  each 
and  where  the  details  of  the  task  investigation  can  be  found  in 
this  report. 

A.  SUBTASK  A 

This  subtask  requested  IDA  to  consider  principal  objectives 
and  priorities  involved  in  the  operation  and  design  of  DTIC’s 
on-line  Scientific  and  Technological  Information  (STI)  data  base 
systems  in  support  of  the  Defense  RDT&E  community.  In  response 
to  this  subtask  we  reviewed  relevant  documentation  including 
regulations  and  directives  and  discussed  the  subject  generally 
with  a  variety  of  DROLS  users.  We  found  a  vast  number  of  regu¬ 
lations  and  directives  have  been  issued  by  the  Department  of 
Defense  and  all  of  the  military  services  that  relate  to  the 
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operation  of  DTIC  and  DROLS  (refer  to  the  References  in  Part  A 
for  a  sample  listing).  In  particular,  descriptive  material  for 
all  ongoing  and  completed  RDT&E  projects  must  be  submitted  to 
DTIC.  The  methods  and  responsibility  for  submitting  this  infor¬ 
mation  are  detailed  in  these  regulations.  There  is  no  doubt 
about  what  should  be  done.  Yet  many  military  agencies  ignore 
them  and  fail  to  submit  the  information  that  they  are  required 
to  submit.  The  net  result  of  this  neglect  is  that  the  DTIC 
files  are  incomplete.  The  amount  of  information  missing  ranges 
up  to  50  percent,  according  to  some  estimates.  It  would  seem 
that  with  such  a  vast  amount  of  information  missing,  DTIC  would 
be  severely  crippled  in  the  performance  of  its  primary  function 
of  being  the  central  depository  and  single  access  point  for  all 
Defense  RDT&E  information.  The  potential  danger  involved  is  that 
the  flow  of  information  is  grievously  impeded,  resulting  in  ig¬ 
norance  of  important  research  findings,  duplication  of  effort, 
possibly  unbuilt  or  poorly  performing  systems  and  greater  cost¬ 
liness  of  U.S.  Defense  RDT&E  efforts.  It  is  clear  that  the  prob¬ 
lem  cannot  be  handled  by  the  present  set  of  regulations.  The 
present  regulations  are  complete;  but  they  are  largely  ignored. 
Unless  some  teeth  can  be  put  into  the  regulations  and  enforce¬ 
ment  encouraged  from  the  top  down,  the  situation  is  likely  to 
continue.  Work  on  this  subtask,  which  was  not  changed,  is  dis¬ 
cussed  further  in  the  first  volume  (Part  A)  of  this  report. 

B.  SUBTASK  B 

This  subtask  originally  required  IDA  to  review  the  current 
secure  communications  system  used  by  DROLS.  We  also  were  re¬ 
quired  to  identify  unclassified  on-line  services  that  could 
access  unclassified  data  and  determine  the  principal  character¬ 
istics  of  communications  services,  including  their  costs.  In 
response  to  this  original  subtask  we  developed  a  model  repre¬ 
senting  a  computer-communications  system.  The  model  provides  a 


method  of  calculating  the  various  principal  characteristics  of 
the  system,  and  is  appended  to  Part  A  of  this  report. 

This  subtask  was  redirected  to  require  a  survey  of  a  sample 
of  DTIC  users  to  obtain  their  reactions  to  the  proposition  that 
DROLS  become  a  completely  unclassified  system.  We  found  an  over¬ 
whelming  majority  of  those  DROLS  users  who  had  classified  ter¬ 
minals  insisting  that  they  remain  so,  while  the  unclassified 
users  were,  by  and  large,  satisfied  with  what  they  could  get 
from  DROLS.  This  survey  is  described  and  the  results  provided 
in  detail  in  Chapter  IV  of  this  volume. 

C.  SUBTASK  C 

This  subtask  initially  requested  us  to  look  at  the  question 
of  using  unclassified  terminals  to  access  a  data  base,  part  of 
which  may  be  classified. 

Since  DTIC  has  been  doing  exactly  this  for  almost  a  decade 
with  no  evidence  that  its  classified  information  has  ever  been 
compromised  by  an  on-line  user  at  an  unclassified  terminal,  the 
question  seems  somewhat  academic.  Moreover,  there  are  now  com¬ 
mercially  available  a  variety  of  "security  kernels"  that  have 
the  capability  of  handling  multi-level  classification  schemes  in¬ 
cluding  such  items  as  the  need-to-know .  However,  these  capabil¬ 
ities  of  commercially  available  software  have  not  been  verified 
and  consequently  this  software  is  not  yet  approved  for  DoD  pur¬ 
poses.  Another  problem  with  any  such  scheme  is  that  the  remote 
terminal  must  be  in  a  properly  secured  area  and  access  to  the 
terminal  itself  must  be  given  only  to  those  who  have  appropriate 
clearances.  If  such  control  of  the  remote  ends  of  the  system 
cannot  be  achieved  then  the  entire  system  could  be  compromised. 
However,  there  seems  to  be  no  real  problem  in  maintaining  the 
system  security  if  the  classified  terminals  are  kept  secure, 
even  though  both  classified  and  unclassified  users  can  access 
the  files . 


Subsequent  to  the  work  reported  above,  subtask  C  was  re¬ 
directed  to  require  that  we  determine  the  need  and  utility  of 
classified  information  as  available  from  DROLS  to  representa¬ 
tive  system  users. 

To  determine  this  need  we  relied  on  the  survey  mentioned 
under  subtask  B  above.  The  general  findings  are  that  classified 
users  need  essentially  complete  information  to  do  their  work 
effectively,  and  they  are  not  willing  to  have  their  search 
results  truncated  because  of  classification.  See  Chapter  IV 
for  more  detail. 

D.  SUBTASK  D 

This  subtask  requested  us  to  examine  the  possible  impact 
of  AUTODIN  II  on  DROLS  if  it  became  the  universal  telecommuni¬ 
cations  medium  for  all  DOD  installations. 

Before  this  task  was  redirected  we  had  the  time  to  give  only 
some  very  preliminary  thought  to  this  question.  The  response 
time  analysis  that  was  done  in  answer  to  part  of  the  original 
subtask  B  described  above  applies  in  part  here  (see  Appendix  D, 
Part  A).  However,  this  analysis  was  abandoned  when  the  task  was 
redirected  since  it  needed  more  data  for  completion. 

The  changed  task  requested  us  to  provide  some  cost  informa¬ 
tion  for  both  classified  and  unclassified  terminals.  We  could 
not  find  any  significant  difference  in  manpower  costs  due  to 
security,  but  there  are  some  differences  in  the  one-time  and 
operating  costs  of  the  two  types  of  terminals.  Costs  are  pre¬ 
sented  in  Appendix  B. 

E.  SUBTASK  E 

This  subtask  requested  us  to  examine  the  possible  impact  of 
office  automation  developments,  including  the  paperless  office 
and  laboratory  to  determine  how  such  developments  may  enhance 


the  functions  of  DROLS.  This  subtask  was  deleted  and  no  further 
report  appears  in  this  paper  beyond  the  following  paragraph. 

Until  the  task  was  redirected  we  managed  to  accomplish  an 
initial  gathering  of  information  and  only  some  very  preliminary 
thinking  about  this  question.  There  did  not  seem  to  be  much 
applicability  to  DROLS  at  present,  but  teleconferencing,  full 
page  image  transmission,  and  systems  based  on  film  or  video 
disc,  perhaps  controlled  by  computer,  may  be  promising  applica¬ 
tions  for  information  processing  and  transfer  to  the  user. 
Generally,  paperless  operations  might  enhance  DTIC  internal  op¬ 
erations  much  the  same  as  they  would  enhance  almost  any  organi¬ 
zation's  internal  operations. 
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THE  COST  OF  HAVING  A  REMOTE  TERMINAL 

We  present  here  three  sets  of  costs,  two  of  which  show  the 
costs  of  acquiring  and  operating  a  classified  terminal,  and  one 
of  which  shows  the  same  for  an  unclassified  terminal.  The  two 
classified  terminals  costs  apply  to  different  times  ...  1969 
and  1980.  In  this  approximately  one  decade,  costs  have  not 
varied  much  despite  the  general  inflation  in  the  economy.  In 
fact,  operating  costs  have  actually  diminished.  The  primary 
reason  for  such  cost  behavior  must  be  attributed  to  the  great 
advances  that  have  been  made  in  the  technology  of  electronic 
data  processing  in  this  period. 

The  1969  cost  data  were  taken  from  a  DDC  report  by 
Richard  K.  Bennertz  that  described  the  various  phases  involved 
in  the  development  of  the  DROLS  (Ref.  B-l) .  The  other  two  sets 
of  costs  are  actual  site  costs  that  might  apply  to  a  user  in 
the  current  time.  These  costs  are  IDA  costs  and  reflect  IDA's 
particular  situation.  IDA  already  had  a  vault  and  the  only  re¬ 
quirement  was  to  find  some  work  space  for  the  terminal,  crypto 
equipment  and  an  operator.  So  site  preparation  costs  were 
fairly  nominal  in  such  a  case.  Also,  monthly  operating  costs 
are  highly  dependent  on  the  user's  proximity  to  DTIC,  which  af¬ 
fects  primarily  the  costs  of  a  dedicated  telephone  line,  or  if  a 
dial-up  mode  is  used  it  will  affect  user  charges.  For  example, 
a  local  call  for  IDA  would  cost  essentially  nothing,  but  a  call 
might  cost  a  substantial  amount  from  a  more  remote  location 
such  as  the  U.S.  West  Coast.  On  the  other  hand  for  an  unclassi¬ 
fied  terminal  the  one-time  costs  are  quite  nominal,  assuming 
the  terminal  will  be  leased  rather  than  purchased,  although  even 
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here,  prices  have  been  coming  down  rapidly  so  that  the  purchase 
of  a  terminal  could  hardly  be  considered  a  major  expense  for 
most  DTIC  users . 

There  are  some  other  costs  involved  in  having  a  classified 
terminal.  These  include  some  share  of  the  security  overhead, 
the  need  for  having  crypto-cleared  individuals  available,  and  an 
operator.  However,  in  no  case  could  we  find  that  these  costs 
added  anything  significant  that  could  be  charged  to  the  terminal. 
For  example,  the  security  force  is  not  likely  to  be  increased 
due  to  a  few  square  feet  of  vault  space  dedicated  to  a  terminal 
in  any  organization  that  has  to  maintain  a  secure  environment 
anyway.  Similarly,  the  individuals  needed  to  operate  the  crypto 
equipment  have  to  be  on  hand  for  other  reasons  and  the  amount 
of  work  the  crypto  requires  involves  a  tiny  fraction  of  their 
time.  The  operator’s  job  is  the  same  whether  or  not  the  terminal 
is  classified.  The  only  distinguishable  additional  costs  of  a 
classified  terminal  are  maintenance  of  the  crypto  equipment  and 
the  monthly  costs  of  the  dedicated  telephone  lines  that  are  nec¬ 
essary  for  classified  terminals.  We  note  that  it  may  be  less 
costly  for  non-local  unclassified  terminals  to  use  a  dedicated 
line,  depending  on  connect  time  and  distance  from  DTIC. 

It  can  be  concluded  that  the  cost  differences  between  clas¬ 
sified  and  unclassified  terminals  are  noticeable.  Whether  they 
are  significant  or  not  depends  on  the  user's  needs  and  attitudes 
toward  achieving  the  desired  result  of  complete  data.  As  we  have 
noted  in  Chapter  IV,  this  result  is  currently  unattainable  any¬ 
way,  even  with  a  classified  terminal,  primarily  because  all  the 
information  that  should  be  in  the  DTIC  files  is  not  there. 

We  note  also  that  there  are  additional  cost  factors  to  DTIC 
(and  to  the  DoD)  in  maintaining  the  classified  DROLS .  Among  these 
are  DTIC  maintenance  of  crypto  equipment,  two  remote  offices  (in 
Boston  and  Los  Angeles  for  local  classified  users)  and  the  non- 
optimum  communications  arrangements  that  ensue  fromthe  requirement 
for  dedicated  lines. 
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COSTS  OF  REMOTE  TERMINALS 


1969  CLASSIFIED  (Ref. 

D- 

1) 

IDA  -  1980*  CLASSIFIED 

One-Time  Costs 

One  Time  Costs 

KG-Crypto  Unit 

$ 

3750  (GFE ) 

Crypto  Aux.  Unit 

$  3000 

Junction  Box 

300 

KG  13-Crypto  Unit 

3448 

(GFE) 

Telephone  Installation 

938 

Telephone  Cable 

Ass  'y 

68 

Site  Preparation 

3800 

Installation 

4000 

A1  lied  Support 

2636 

Site  Preparation 

750 

Installation  Team 

1037 

Safe  (Including 

Shipping) 

3050 

TOTAL 

$12461 

TOTAL 

$10868 

Monthly  Operating  Costs 

Monthly  Operating 

Costs  (1980) 

Termi nal 

895 

Mai ntenance 

167 

Modem  Circuit 

566 

Telephone  Line 

181 

Modem  Lease 

250 

Terminal  & 

Peri pheral s  Lease 

418 

TOTAL 

1461 

TOTAL 

$1016 

1980  UNCLASSIFIED  SYSTEM 

One-Time  Costs 

Monthly  Operating 

Costs 

Installation  $70 

Mai ntenance 

145 

Telephone  (Time 

&  Distance  Dep.) 

0 

TOTAL 

$145 

*Although  the  IDA  terminal  was  installed  in  1979  as  a  classified 
terminal,  we  believe  that  these  costs  are  reasonably  close  to 
actual  1980  costs. 
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SURVEY  OF  DROLS  USERS 

As  reported  in  Chapter  V  of  this  report,  surveys  were  con¬ 
ducted  at  a  total  of  eleven  user  sites,  including  IDA.  In  pre¬ 
paration  for  this  survey,  the  survey  questions  were  sent  to  the 
potential  respondents  in  advance,  along  with  some  background  in¬ 
formation  and  a  cover  letter  signed  by  the  DTIC  Administrator. 
This  Appendix  presents  all  this  material  in  its  original  form 
except  for  the  footnote  on  page  C-7,  which  has  been  added  here 
to  indicate  a  change  that  we  learned  about  after  the  survey  had 
been  completed. 
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■’iLEXANDPIA,  V I  n  G I  IVi  *  A  1.21  • 


DTIC-I 


1  August  1980 


Gentlemen: 

As  an  effort  to  reassess  and  improve  our  technical  information  services, 
DTIC  has  contracted  with  the  Institute  for  Defense  Analyses  (IDA)  to  help 
us  evaluate  possible  ways  to  improve  the  overall  effectiveness  and 
efficiency  of  the  Defense  RDT&E  On-Line  System  (DROLS).  Your  organization 
has  been  selected  as  part  of  a  small  sample  of  DROLS  users  from  which  to 
gather  information  about  DROLS  usage.  Enclosed  please  find  four  copies  of 
a  questionnaire  that  IDA  has  prepared.  We  ask  that  you  have  three  members 
of  your  organization’s  staff  examine  them.  The  three  persons  should  be 
selected  as  follows: 

a.  A  technical  information  services  person  familiar  with  DROLS. 

b.  A  technical  project  manager  who  supervises  bench-level  personnel. 

c.  A  bench-level  technical  person  whom  you  know  uses  DROLS. 

In  a  few  days,  an  IDA  staff  member  will  contact  you  to  conduct  interviews 
of  the  individuals  you  have  selected.  The  interview  will  focus  on  the 
questions  in  the  enclosed,  which  are  presented  here  so  that  you  will  have  a 
preview  of  the  scope  of  the  questions  and  be  able  to  do  some  preliminary 
thinking  about  the  answers. 

I  would  appreciate  your  cooperation  in  this  matter  and  in  assisting  IDA  to 
accomplish  their  assigned  effort. 


Sincerely 


1  Enel 


HUBERT  E.  SAUTER 
Admini s  t  r a  t  o r 


QUESTIONNAIRE  FOR  DROLS  USERS 


I .  INTRODUCTION 

Under  a  contract  with  the  Defense  Technical  Information 
Center  (DTIC),  the  Institute  for  Defense  Analyses  (IDA)  has 
been  tasked  with  evaluating  a  possible  change  in  the  DTIC- 
operated  Defense  Research  On-Line  System  (DROLS).  Your  faci¬ 
lity  accesses  bibliographic  information  from  DTIC.  We  have 
elected  to  query  you  about  attitudes  toward  the  use  of  DTIC 
services  because  you  are  among  the  heavier  users  of  DROLS. 

In  the  following  paragraphs  we  shall  describe  several  pro¬ 
grammed  changes  in  DROLS  and  then  we  shall  describe  a  possible 
further  change.  We  should  like  to  have  your  comments  on  this 
change,  in  accordance  with  a  set  of  questions  that  follows  the 
descriptions . 

II  .  PROGRAMMED  CHANGES  IN  DROLS 

Three  changes  in  DROLS  are  to  be  implemented  this  year 
(1980).  These  changes  will  affect  only  users  who  have  unclas- 
sified  terminals.  The  first  change,  implemented  in  July  1980, 
permits  the  user  of  an  unclassified  terminal  to  access  unclas¬ 
sified  entries  in  the  data  file  for  technical  reports.*  That 
is,  unclassified  entries  (the  only  classifiable  fields  in  an 
entry  are:  titles,  descriptors,  identifiers,  and  abstracts) 

* 

While  there  are  four  files  in  DROLS,  the  examples  of 
improved  access  are  stated  in  terms  of  Technical  Report 
file. 


will  become  available  to  an  unclassified  terminal.  About  90% 
of  the  Technical  Reports  and  their  entries  are  unclassified. 

Of  the  approximately  10%  of  the  Technical  Reports  that  are 
classified,  31%  have  completely  unclassified  entries.  The 
latter,  except  for  some  abstracts  which  have  limited  distribu¬ 
tion,  will  be  fully  accessible  to  an  unclassified  user.  In 
other  data  banks,  the  proportion  of  classified  citations  (cita¬ 
tions  are  the  equivalents  of  Technical  Report  entries)  is 
even  lower  than  that  of  the  Technical  Reports,  so  an  even 
higher  proportion  of  these  data  banks  is  available  to  unclassi¬ 
fied  terminals.*  If  a  classified  report  or  entry  (of  which  the 
existence  has  been  determined  by  searching  the  technical  Ab¬ 
stract  Bulletin  or  DROLS)  is  wanted  by  a  searcher,  it  can  be 
requested  on-line  or  by  mail  and  it  will  be  mailed  by  DTIC  in 
an  average  of  3-^  days.** 

The  second  change,  to  be  implemented  in  October  1980,  will 
be  the  initiation  of  a  dial-up  capability  for  any  unclassified 
terminal  user  registered  with  DTIC.  With  this  system  the  user 
can  access  DTIC  with  almost  any  type  of  terminal  and  telephone 
line  or  TYMNET.  This  change  will  reduce  substantially  the  cost 
of  accessing  DROLS. 

The  third  change,  to  be  implemented  in  about  November  1980, 
attempts  to  overcome  a  major  deficiency  in  the  first  change 
noted  above.  All  unclassified  fields  from  a  classified  entry 
will  be  accessible  to  an  unclassified  DROLS  user;  that  is, 
unclassified  titles,  descriptors,  identifiers,  and  abstracts. 


Access,  similar  to  that  to  be  made  in  July  1980  to  the 
Technical  Report  Pile,  is  already  available  to  the  Work 
Unit  and  Program  Planning  Files.  The  IR&D  file  is  acces¬ 
sible  only  to  government  personnel  since  it  contains  pro¬ 
prietary  data. 

We  made  no  estimate  for  the  time  required  for  delivery  of 
registered  U.S.  mail  and  for  the  internal  distribution  of 
mail  at  the  receiving  organization. 


Only  certain  limited  distribution  abstracts  may  not  be  avail¬ 
able  to  all  subscribers.  For  this  planned  change  the  classi¬ 
fied  user  and  terminal  would  perceive  no  difference  from  the 
current  system. 

Unclassified  terminals  would  be  capable  of  receiving  rela¬ 
tively  complete  bibliographic  information.  We  estimate  that 
about  7%  of  the  entries  for  all  technical  reports  ( 69 %  of  the 
entries  for  classified  technical  reports)  will  have  from  one 
to  four  fields  (title,  descriptors,  identifiers,  or  abstracts) 
that  will  not  be  accessible  to  unclassified  terminals.  Never¬ 
theless,  a  searcher  at  an  unclassified  terminal,  in  response 
to  a  subject  search,  would  be  provided  at  least  an  AD  number, 
organization  author,  personal  author,  and  report  date.  There¬ 
fore,  such  a  searcher  will  know  that  a  classified  report  exists 
that  matches  the  inquiry.  An  exception  to  this  exists:  No 
find  will  be  reported  on-line  if  the  searcher  performs  a  sub¬ 
ject  search  and  the  matching  term  in  an  entry  is  classified  or 
is  in  a  classified  field.*  If  an  unclassified  terminal  user 
wants  a  report  or  classified  entry  found  by  a  search,  it  must 
be  ordered  for  delivery  by  mail.  If  a  comprehensive  search 
including  classified  entries  is  wanted,  an  off-line  biblio¬ 
graphy  must  be  ordered.  Again,  we  emphasize  that  the  informa¬ 
tion  missing  from  the  on-line  system,  in  any  case,  is  only 
bibliographic  information.  Reports  or  other  documents  must 
still  be  requested  and  delivered  by  mail. 

Distribution  limitations  would  still  apply.  Therefore, 
except  for  classification  restrictions,  almost  all  abstracts 
would  be  available  to  DoD  users,  fewer  to  other  U.S.  government 
users,  and  still  fewer  to  DoD  contractors,  as  is  the  case  today. 

* 

We  have  been  informed  by  DTIC  that  this  exception  will  no 
longer  exist  due  to  a  modification  planned  to  the  system 
later  this  year.  Unfortunately,  we  received  this  informa¬ 
tion  after  the  survey  was  completed. 


Hereafter,  we  shall  refer  to  the  DROLS  with  the  above 
three  changes  as  the  "Baseline"  System. 

III.  POSSIBLE  CHANGE  IN  DROLS 

The  classified  on-line  component  of  the  Baseline  System 
would  be  discontinued. 

The  unclassified  on-line  component  of  the  Baseline  System 
would  continue.  Those  title,  identifier,  or  abstract  fields 
that  are  classified  would  not  be  accessible.  The  descriptor 
field  would  be  improved  to  contain  all  the  posting  terms  (from 
the  DRIT)  that  are  applicable  to  the  report  so  that  subject 
searches  would  be  more  successful  than  searches  from  an  unclas¬ 
sified  terminal  with  the  Baseline  System.  As  in  the  Baseline 
System,  if  a  user  would  want  a  classified  entry  it  could  be 
ordered  for  delivery  by  mail. 

IV.  QUESTIONNAIRE 

We  plan  to  visit  or  phone  you  for  a  personal  interview  at 
which  time  you  will  be  asked  to  answer  the  questions  orally. 

The  purpose  of  this  questionnaire  is  to  give  you  some  indica¬ 
tion  of  the  scope  of  the  questions  and  to  give  you  time  to 
think  about  the  answers. 

1.  Please  identify  yourself  as: 

a.  Information  Specialist  (e.g.,  librarian) 

b.  Researcher 

c.  Research  manager 

2.  Estimate  your  use  of  DROLS  over  the  last  year 

a.  Daily 

b.  Weekly 

c.  Monthly 

d.  Yearly 

e.  Never 
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•  3. 


What  proportion  of  the  documents  (not  books)  in  your 
collection  is  classified? 


4.  Have  you  used  commercial  on-line  bibliographic  services? 

a.  Yes 

b.  No 


5.  If  you  have  only  unclassified  terminals,  why  do  you  not  have 
a  classified  terminal?  (Use  as  many  answers  as  apply.) 

^  a.  No  need  for  classified  information 

b.  Little  need  for  classified  information 

c.  Cost 

d.  Terminal  procurement  difficulties 

e.  Secure  facility  difficulties 

f.  Necessary  information  is  available  in  other  ways 

g.  Other  (comment) 

•  6.  If  you  have  a  classified  terminal,  on  what  basis  do  you 

justify  the  additional  expense  and  provisions  over  that 
for  an  unclassified  terminal?  (Use  as  many  answers  as 
apply. ) 


a.  Necessity  for  classified  information 

b.  Response  speed  for  classified  information 

c.  Cost  not  a  problem 

d.  Terminal  provisions  not  a  problem 

e.  Other  (comment) 


7.  Would  you  consider  changing  your  terminal  to  unclassified 
given  the  changes  in  DROLS  programmed  for  1980? 

a .  Yes 
a .  No 


8.  How  was  the  capital  (equipment)  expense  of  your  terminal (s) 
funded? 


9< 

10, 


11, 

12 


a.  By  our  library 

b.  By  our  organization  (not  library) 

c.  Contract  or  Project 

d.  Other  (How?) 

Do  you  know  how  much  It  cost  Installed? 

How  is  the  operational  expense  of  your  terminal (s)  funded? 

a.  By  our  library 

b.  'By  our  organization  (not  library) 

c.  Contract  or  Project 

d.  Other  (How?) 

Do  you  know  how  much  it  costs? 

If  only  unclassified  service  were  provided,  what  would  you 
most  likely  do  about  the  missing  classified  fields  (some 
titles,  descriptors,  identifiers,  or  abstracts)? 

a.  Nothing 

b.  Order  the  classified  entries 

c.  Order  the  classified  report 

Referring  to  Question  12,  on  what  basis  would  you  take 
actions  12a,  12b,  or  12c? 

a.  Ignore  the  report;  inadequate  information  and  too 
much  delay  in  other  options 

b.  Have  enough  information;  no  more  needed 

c.  Require  entry  information  to  proceed 

d.  Might  as  well  order  report 

e.  Other  (comment) 

14.  Would  you  be  satisfied  with  an  unclassified  DROLS? 

a.  Strong  yes 

b.  Moderate  yes 

c .  Indifferent 

d.  Moderate  no 

e.  Strong  no 


13, 


C-10 


15-  Why? 


a.  No  strong  need  for  classified  bibliographic  information 

b.  Need  classified  information,  but  can  function  adequately 
without  classified  DROLS 

c.  Cost  of  classified  terminal  is  too  high 

d.  Can  use  DTIC  searches 

e.  Difficult  to  provide  for  classified  terminal 

f.  Strong  need  for  classified  information 

g.  Classified  search  faster  with  classified  terminal 

h.  Search  more  fruitful  when  performed  in-house  rather 
than  by  DTIC 


16. 


What  modification  to  DROLS  (not  discussed  heretofore)  do 
you  think  would  be  valuable? 
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